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FOREWORD 


The  Fort  Hood  Field  Unit  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  conducts  research  and  development  to 
promote  the  effective  combat  use  of  the  soldier  and  to  improve  soldier-machine 
interfaces  in  future  system  designs.  Data  are  reported  here  from  the  Sgt 
York  Follov-On  Evaluation  I  (FOE  I),  which  generated  objective  performance 
measures  on  individual  soldier,  team,  and  system  performance  in  the  execution 
of  the  target  engagement  sequence.  The  implications  of  these  data  for 
developing  future  air  defense  systems  are  discussed  for  such  areas  as  system 
and  subsystem  performance,  tactical  performance,  individual  and  crew 
performance,  personnel  factors,  and  training. 

The  primary  objective  of  this  effort  was  to  structure  the  data  into  a 
human  performance  data  base  that  could  be  incorporated  into  computer  models  of 
human  and  system  performance  in  future  Forward  Area  Air  Defense  Systems 
(FAADS).  Further,  the  data,  the  analyses,  and  the  recommendations  in  this 
report  provide  information  directly  relevant  to  consideration  of  the  soldier 
into  future  FAADS  design. 

The  participation  of  ARI  in  the  Sgt  York  FOE  I  was  in  accordance  with  a 
Letter  of  Agreement  between  ARI  and  the  U.S.  Operational  Test  and  Evaluation 
Agency  (OTEA)  dated  15  June  1983.  The  preparation  of  this  report  was  funded 
under  Research  Task  114,  Air  Defense  Crew  and  Operator  Performance,  sponsored 
by  the  Air  Defense  School.  The  effort  was  in  accordance  with  a  Letter  of 
Agreement  between  ARI  and  the  U.S.  Army  Air  Defense  Artillery  School  dated  20 
September  1986.  The  data  and  analyses  presented  in  this  report  have  been 
provided  to  OTEA  and  the  U.S.  Army, Air  Defense  School. 
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HUMAN  FACTORS  PERFORMANCE  DATA  FOR  FUTURE  FORWARD  AREA  AIR  DEFENSE  SYSTEMS 
(FAADS) 


EXECUTIVE  SUMMARY 


Requirement: 

Considerable  data  were  collected  during  the  Sgt  York  Follow-On  Evaluation 
I.  When  system  acquisition  was  cancelled  by  the  Secretary  of  Defense,  much  of 
the  processing  of  the  collected  data  was  also  cancelled  or  brought  to  a  hasty 
conclusion.  The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  proposed  that  additional  analyses  be  performed  and  reported. 
This  document  reports  the  findings  of  this  effort.  It  documents  the  collec¬ 
tion  and  analysis  of  man-system  performance  events  during  271  target  engage¬ 
ment  sequences  and  answers  the  requirement  to  develop  information  of  use  in 
the  design  of  future  forward  area  air  defense  systems. 


Procedure: 

The  principal  data  for  this  analysis  was  produced  by  an  automated  digital 
data  bus  during  the  Sgt  York  force-on-force  trials  (nonlive  fire)  that  took 
place  at  Fort  Hunter-Liggett  in  1985.  Printouts  were  examined  and  a  criterion 
was  developed  for  usable  engagements  (events  that  proceeded  all  the  way 
through  to  a  trigger  pull).  Using  this  criterion,  data  from  271  target  en¬ 
gagement  sequences  occurring  during  15  of  the  29  valid  trials  were  included  in 
the  analysis.  Collateral  data  were  assembled  relating  to  the  independent 
variables  associated  with  the  engagements. 


Findings: 

The  data  yielded  findings  on  many  issues  related  to  the  design  of  FAADS. 
Objective  performance  data  are  examined  with  respect  to  fourteen  issues: 

•  Actual  system  performance  compared  with  system  performance 
requirements 

•  The  negative  impact  of  some  semi -automation 

•  Variations  in  fire  unit  performance 

•  The  impact  of  various  tactical  scenarios  on  performance 

•  The  effect  of  hostile  rotary-wing  tactics  on  system  performance 

•  The  effect  of  various  electronic  countermeasure  (ECM)  conditions 

•  Varying  performance  levels  as  a  function  of  the  time  of  day 

•  Reaction  to  first  target  appearance  at  various  ranges 

•  Specific  task  performance  of  gunner  and  squad  leaders 

•  The  effect  of  wearing  mission-oriented  protective  posture  gear 
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•  Varying  target  vorkloads  on  the  operator 

•  Characteristics  of  crewmen  related  to  individual  performance 

•  The  influence  of  "mental  category"  on  crew  mix  and  crew  performance 

•  Crew  member  training,  past  experience,  and  performance 


Utilization  of  Findings: 

This  report  provides  suggestions  for  application  of  the  performance  data 
to  soldier-performance  models  that  will  impact  on  decisions  relating  to  future 
FAADS.  Recommendations  are  also  made  for  Improvement  of  future  FAADS 
evaluations. 
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HUMAN  FACTORS  PERFORMANCE  DATA  FOR  FUTURE 
FORWARD  AREA  AIR  DEFENSE  SYSTEMS  (FAADS) 

Introduction 


Future  FAADS 

Whatever  the  future  of  specific  forward  area  air  defense  artillery 
systems,  it  would  appear  to  be  a  safe  prediction  that  many  new  systems  are 
going  to  be  developed  and  evaluated.  Whether  the  systems  are  soldier-portable 
(e.g..  Stinger  and  AAWS-M)  or  multi-crew  self-propelled  vehicles  (e.g.,  ADATS, 
Shahine,  or  the  advanced  Rapier),  some  combination  of  air  defense  artillery 
systems  will  emerge  in  the  future  to  meet  the  continuing  and  Increasing  threat 
that  hostile  fixed-wing  and  rotary  aircraft  bring  to  the  battlefield.  While 
some  of  these  may  be  non-developmental  items,  it  appears  safe  to  predict  that 
much  new  (and  better)  design  will  be  needed  for  future  FAADS. 

Estimates  of  the  requirements  from  the  battlefield  continue  to  increase  in 
severity.  Effective  weapon  ranges  seem  to  expand  constantly,  perhaps  up  to  10 
kilometers.  The  most  Important  requirement,  however,  is  response  time.  We 
are  dealing  with  system  response  times  of  less  than  10  seconds.  Both  in  speed 
and  in  tactics,  hostile  fixed-wing  and  rotary  aircraft  will  be  directed  to 
provide  decreased  exposure  which  in  turn  will  require  air  defense  artillery 
response  times  to  decrease. 


Role  of  the  Human  Operator 

With  very  short  system  response  times  required,  the  role  of  the  human 
operator  in  FAADS  must  be  questioned.  While  there  are  many  potential  uses  for 
the  human  operator  in  air  defense  systems,  there  may  not  be  time  for  the  human 
to  react.  Or,  if  he  is  in  the  control  loop,  his  actions  must  be  very  simple 
ones. 

For  some  years  this  problem  has  been  increasingly  evident  in  the  design  of 
air  defense  systems,  and  there  have  been  more  and  more  attempts  to  introduce 
system  and  subsystem  automation.  Briefly,  the  trend  has  been  first  to  change 
manual  tasks  into  semi-automatic  tasks  and  then  to  provide  support  with 
automatic  decision-aiding  systems.  An  example  of  the  first  is  manual 
computer-assisted  firing;  an  example  of  the  second  is  automated  Identification 
Friend  or  Foe  (IFF).  Some  data  from  the  SGT  York  testing  suggests  that  the 
particular  automation  Introduced  may  not  have  helped  system  performance.  The 
particular  technique  of  automation  may  have  reduced  system  capability  by 
Increasing  the  system's  response  time. 

Operational  tests  over  the  past  five  years  have  suggested  that  the  soldier 
has  been  used  in  less  than  optimal  ways  in  air  defense  systems.  Beyond  poor 
basic  human  engineering  design  for  the  crew  stations  (Babbitt,  1987),  some  of 
the  fundamental  tasks  being  performed,  for  example,  by  the  gunner  and  squad 
leader  appear  to  be  badly  designed,  particularly  when  various  levels  of 
automation  are  Introduced. 
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Computer  Simulation  for  Design 

Part  of  the  problem  appears  to  be  that  operational  tasks  and  roles  for  the 
soldier  are  inadequately  tested  in  development.  Insufficient  time  and 
resources  are  applied  to  verify  the  feasibility  and  timeliness  of  required 
human  task  performance.  It  is  to  be  hoped  that  in  the  future  more  attention 
is  given  by  system  design  teams  to  physical  simulation  and  empirical  check  of 
man's  roles  before  system  designs  are  frozen. 

There  is  another  method  that  could  be  used  effectively  to  predict  and 
design  human  operator  tasks  in  air  defense  systems.  That  method  is  the 
development  of  quantitative  human  performance  models  through  computer 
simulation  (Chubb,  Laughery,  &  Prltsker,  1987).  Briefly,  a  number  of 
extensively  developed  and  proven  models  are  available  for  simulating  human- 
system  performance  and  predicting  quantitatively  (and  accurately)  expected 
levels  of  human  performance  in  complex  systems. 

Such  simulations  have  been  widely  and  successfully  used  in  Army  weapon 
systems  design.  For  example,  the  ARI  Crew  Performance  Model  (Schwalm, 

Crumley,  Coke,  &  Sachs,  1981)  has  recently  been  used  to  predict  soldier-system 
performance  in  advanced  concepts  for  the  Howitzer  Improvement  Program.  These 
models  have  also  been  developed  specifically  for  man-machine  problems  in  air 
defense.  One  specific  example  was  MOPADS  (Models  of  Operator  Performance  in 
Air  Defense  Systems)  based  on  the  SAINT  modeling  language  and  tested  on  the 
AN/TSQ-73  system  and  the  IHAWK.  (Polito  &  Laughery,  1984).  The  technical 
methodology  is  well  developed  and  tested  (Rouse,  1980). 

In  essence,  if  a  specific  model  is  not  available  for  the  particular 
system,  the  following  steps  have  to  be  taken  for  a  quantitative  man-machine 
model : 

•  The  proposed  system  and  alternatives  must  be  described  at  a  suffi¬ 
cient  level  of  detail  so  that  all  critical  system  tasks  are  identified. 

•  An  appropriate  computer  model  framework  is  selected  to  represent  the 
system  and  its  functions.  This  is  a  simulation  of  the  system,  and  will  be 
successful  to  the  degree  to  which  the  simulation  maps  into  the  system.  Many 
models  and  flexible  simulation  languages  are  available  for  use.  Extreme 
precision,  incidentally,  is  not  necessarily  mandatory  depending  upon,  among 
other  things,  what  level  of  accuracy  is  required  in  quantitative  predictions. 
For  design  feasibility  studies,  it  is  often  sufficient  to  identify  that  a 
design  alternative  is  an  acceptable  candidate  for  advanced  development. 

•  Data  estimates  have  to  be  available  to  simulate  predicted  performance 
values  that  might  be  expected  for  each  system  and  subsystem  parameter  in  the 
model.  These  may  be  data  distributions  or  they  may  be  single  data  points 
depending  upon  the  particular  model  framework.  These  are  the  data  that  the 
model  will  use  to  compute  predicted  performance  levels. 


Performance  Data  For  Modeling 

Data  for  the  model  may  be  generated  in  a  number  of  ways.  First,  for  some 
parameters  there  are  very  substantial  data  in  the  literature.  In  fast 
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response  time  systems  where  operator  reaction  time  is  critical,  thousands  of 
published  data  points  are  available  for  human  reaction  time  data  distribu¬ 
tions.  Second,  no  known  data  may  exist,  and  estimation  techniques  (some  of 
which  are  quite  satisfactory)  must  be  used. 

Third,  data  may  come  from  empirical  situations  which  have  another 
objective.  For  example,  developmental  and  operational  tests  of  specific 
systems  can  provide  data  that  go  beyond  the  particular  and  can  be  used  in  far 
more  general  or  other  specific  contexts.  This  report  supplies  data  from  just 
such  a  source.  In  1985,  a  very  extensive  operational  test  was  conducted  with 
the  Sgt  York  air  defense  system.  Very  large  amounts  of  performance  data  were 
collected  from  air  defense  tasks  performed  by  the  crews  in  the  Sgt  York.  Of 
particular  importance  was  data  on  target  engagement  events  during  271 
operational  trials.  From  this,  the  human  and  system  performance  data  base 
presented  in  this  report  was  developed .  These  data  are  applicable  beyond  the 
Sgt  York,  and  are  analyzed  in  depth  in  this  report. 


Purpose  of  this  Report 


Building  from  the  Sgt  York  target  engagement  sequence  data  base  and  the 
objective  human  and  system  performance  data  there,  the  following  objectives 
are  the  purpose  of  this  report: 

•  Developing  specific  recommendations  for  the  use  of  the  human  operator 
in  future  FAADS  with  particular  emphasis  on  future  automation. 

•  Generating  data  on  a  wide  variety  of  potential  air  defense  systems 
variables  in  such  areas  as  system  and  subsystem  performance,  tactical  perfor¬ 
mance  variations,  individual  and  crew  performance,  personnel  quality,  and 
training.  The  present  findings  suggest  which  variables  are  important 

and  which  might  not  be  Important. 

•  Generating  significant  parameters  and  data  for  computer  models  of 
human  performance  in  future  FAADS  design. 

•  Providing  data,  concepts,  and  awareness  for  the  critical  role  of  the 
soldier  in  air  defense  systems  and  his  better  utilization  in  future  FAADS. 
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Human  Performance  Data 


The  FAADS  Target  Engagement  Sequence  (TES) 

The  target  engagement  sequence  is  at  the  heart  of  the  air  defense  mission. 
A  generic  model  is  presented  in  this  chapter  to  describe  the  target  engagement 
sequence  which  would  apply  to  any  forward  area  air  defense  weapon  system.  The 
generic  model  is  then  expanded  and  refined  so  that  it  applies  to  specific  FAAD 
systems  for  ADATS,  Liberty  (Shahine),  Tracked  Rapier,  and  Paladin  (Roland). 
Further  amplification  of  the  target  engagement  sequence  includes  a  model 
specific  to  the  Divisional  Air  Defense  Gun  System  (D1VAD),  and  its  several 
acquisition  system  modes  of  operation.  Integration  of  the  specific  target 
engagement  sequences  with  the  generic  model  will  be  discussed. 


Generic  Target  Engagement  Sequence 

In  the  simplest  terms,  target  engagement  involves  detecting,  identifying, 
and  firing  on  a  target.  That  last  segment  has  long  been  machine  supported, 
whether  such  support  means  firing  a  bullet,  a  shell,  or  a  missile.  More 
recently,  machines  have  helped  system  operators  to  aim  at  the  target,  to  lead 
the  target  appropriately,  and  to  track  and  identify  the  target. 

Now,  in  a  variety  of  FAADS  applications,  many  of  the  target  engagement 
tasks  are  machine  aided.  Radar  systems  can  help  to  detect  targets  as  well  as 
to  discriminate  friends  from  foes.  The  type  of  weapon  system  used  against  a 
target  has  an  impact  on  the  actions  that  must  be  taken  by  human  operators  and 
the  help  that  machine  systems  will  provide.  Attacking  a  plane  using  a 
shoulder-fired  Stinger  weapon  places  very  different  demands  on  an  operator 
than  does  using  a  weapon  system  such  as  the  Sgt  York,  which  is  capable  of 
performing  more  functions  automatically. 

The  FAADS  Target  Engagement  Sequence  (TES)  will  vary  in  its  particulars 
from  one  weapon  system  to  another,  but  there  will  also  be  important  parallels 
within  the  various  systems.  Whatever  the  system,  a  target  must  be  detected, 
it  must  be  acquired  by  the  weapon,  and  it  must  be  fired  at.  A  possible 
sequence  of  steps  leading  from  target  appearance  through  cease  fire  is 
indicated  below: 

1.  Target  appears. 

2.  Target  is  detected. 

3.  Target  is  identified  as  foe. 

4.  Target  is  tracked. 

3.  Target  elevation,  azimuth,  and  range  are  determined. 

6.  Aiming  point  is  determined,  i.e.,  firing  solution  is  calculated. 

7.  Weapons  system  is  aimed  at  target. 

8.  Decision  to  fire  is  made. 

9.  Firing  begins. 

10.  Firing  ceases. 


4 


Some  variation  may  occur  among  systems  as  to  the  order  in  which  the  steps 
of  the  target  engagement  sequence  must  be  taken;  still,  some  steps  must  be 
accomplished  before  other  steps  are  taken.  For  example,  the  target  must  be 
detected  before  any  of  the  rest  of  the  sequence  can  proceed.  It  is  Important 
to  classify  the  target  as  foe  before  firing  on  it,  but  that  classification  can 
precede  or  follow  the  determination  of  the  exact  location  of  the  target. 

The  information  obtained  from  the  fifth  step  is  needed  for  the  sixth 
step,  and  certainly  the  weapon  system  should  be  aimed  (Step  7)  before  firing 
begins  (Step  9).  In  other  words,  although  there  is  some  interdependence  among 
the  steps  in  the  target  engagement  sequence,  variations  in  the  order  of 
occurrence  are  possible.  These  variations  are  dependent  on  (1)  the  specific 
total  weapon  system  and  the  way  the  tasks  are  allocated  between  human  operator 
and  machine  components  for  that  system,  and  also  on  (2)  the  particular  way 
that  a  given  target  engagement  sequence  develops. 

To  locate  a  target  precisely  enough  to  be  able  to  hit  it,  elevation, 
azimuth,  and  range  Information  is  needed  and  should  be  obtained  before  the 
target  is  fired  on.  A  human  operator  can  establish  elevation  and  azimuth  by 
superimposing  some  sighting  element  (cross  hairs,  reticle,  etc.)  over  the 
target.  Range  information  is  less  easily  obtained.  Radar  and  laser  are  two 
ways  of  getting  range  data.  A  moving  target  presents  special  difficulties 
because  its  position  is  constantly  changing.  To  hit  it,  its  position  must  not 
only  be  determined  but  predicted  so  that  the  weapon  system  can  intercept  it. 

To  provide  an  overview  for  this  discussion,  a  model  is  presented  which 
Illustrates  the  FAADS  generic  target  engagement  sequence  functions  (see 
Figure  1).  As  will  be  seen  with  the  description,  function,  and  flow  diagrams 
for  specific  systems,  each  system  will  be  different  as  weapon  systems,  yet 
will  be  subsumed  under  the  generic  target  engagement  sequence. 

The  U.S.  Army's  comprehensive  FAAD  plan  consists  of  five  different 
elements:  Line-of-Sight,  Forward-Heavy  (LOS-FH);  Line-of-Sight ,  Rear  (LOS- 
R);  Non-Line-of-Sight;  Combined  Arms;  and  Command-Control  Information  (C2I) 
("After  DIVAD,  an  $ll-billion  plan,”  ).  LOS-FH  is  considered  the  Sgt  York 
replacement.  An  Interim  system  of  European  design  will  be  fielded  for  the 
line-of-sight  forward  defense  of  heavy  divisions.  LOS-FH  is  to  be  a  combined 
missile  and  gun  system.  LOS-FH  armored  vehicles  are  to  travel  along  with 
tanks  and  other  armored  vehicles  in  the  forward  half  of  the  Army's  heavy 
divisions.  Under  congressional  mandate,  a  LOS-FH  replacement  for  Sgt  York  was 
to  be  selected  by  November,  1987,  with  a  prior  shoot  off.  An  existing  system 
currently  in  production  was  to  be  selected  in  November,  and  fielded  by  1990. 
The  competitors  for  LOS-FH  were:  (1)  ADATS  system  from  Martin-Marietta 
Orlando  Aerospace  and  Oerlekon  Aerospace  of  Canada;  (2)  Liberty  (evolved  from 
the  French-made  Shahlne  system)  from  LTV  Missile  and  Electronics  Group's  Mis¬ 
sile  Division  and  Thomson-CSF  of  France;  (3)  Rapier  from  United  Technologies, 
FMC  Corp.,  and  British  Aerospace;  and  (4)  Paladin  (a  renamed  version  of  the 
Roland)  from  Hughes  Aircraft,  Messerschmitt-Boeoklu-Blohm  of  West  Germany,  and 
SNI  Aerospatiale  of  France  (Adams,  1987).  All  four  systems  have  self- 
contained  radars  for  locating  targets  and  missiles  with  a  range  of  at  least  6 
km. 
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Figure  1 


Forward  Area  Air  Defense  Systems  (FAADS)  generic  target 
engagement  sequence  functions. 
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Air  Defense,  Anti-Tank  Systems  -  ADATS.  ADATS  is  a  day/night  and  adverse 
weather  missile  system  capable  of  engaging  air  targets  or  tanks.  Each  fire 
unit  is  self-contained,  and  uses  its  own  radar  for  early  detection  of  targets. 
The  fire  unit  can  be  incorporated  into  a  network  of  layer  radars  for  command 
and  control.  The  target  engagement  sequence  used  for  ADATS  is  shown  in  Figure 
2.  Characteristics  of  the  ADATS  are  presented  in  Table  1. 

Liberty  (Evolution  of  the  Shahine).  The  Liberty  originally  evolved  from 
the  Crotale  air  defense  system  which  was  a  forerunner  of  the  Shahine  air 
defense  system.  The  system  is  configured  with  2,  3,  or  4  fire  units,  and  one 
or  two  acquisition  and  coordination  units.  The  modular  system  can  be  mounted 
on  a  tank  chassis,  wheeled  armored  vehicle,  or  air  transportable  shelters. 
Liberty  is  a  surface-to-air  defense  system  designed  for  all-weather  intercep¬ 
tion  of  targets.  The  target  engagement  sequence  used  for  the  Liberty  is 
presented  in  Figure  3.  Characteristics  of  the  Liberty  are  provided  in  Table  1 
from  a  description  of  the  Shahine  air  defense  system.  Liberty  is  an  evolution 
of  the  French-made  Shahine  system  used  in  Saudi  Arabia  (the  system  consists  of 
a  25  mm  gun  and  12  missiles  on  an  Abrams  chassis)  (Adams,  1987). 

Tracked  Rapier.  An  armored  version  of  a  cargo  carrier  is  being  used  as 
the  launch  vehicle  for  Tracked  Rapier.  The  weapon  system  is  highly  mobile  and 
is  able  to  defend  mobile  forces  against  low-flying  fixed-wing  and  rotary-wing 
aircraft.  The  system  carries  eight  missiles,  four  on  each  side.  The  missiles 
can  be  fired  and  guided  at  low  elevations  over  the  cab.  The  armored  cab  is 
designed  to  be  operated  by  crew  members  in  MOPP  gear.  The  Tracked  Rapier  has 
day/ night  capability.  Optical  tracking  facilities  are  combined  with  thermal 
imaging.  The  system  may  operate  in  a  passive  mode,  and  has  automatic  passive 
search  when  the  radar  is  switched  off.  The  operator  may  select  optical  or  IR 
tracking.  The  target  engagement  sequence  for  the  Tracked  Rapier  is  presented 
in  Figure  4.  Characteristics  of  the  Tracked  Rapier  are  identified  in  Table  1. 

Paladin  (Renamed  Version  of  the  Roland  II).  Paladin  is  an  all- 
weather/clear-weattier  surface-to-air  weapon  system.  It  was  designed  for 
defence  against  very  low,  low,  and  medium  altitude  fixed-wing  and  rotary¬ 
wing  targets.  The  weapon  system  can  be  incorporated  into  various  armored 
vehicles,  and  is  able  to  function  as  an  Independent  unit.  The  fire  unit  can 
be  mounted  on  a  tracked  vehicle  or  on  a  truck.  The  Roland  (Paladin)  is  in  its 
third  updated  version,  and  has  increased  range,  speed,  and  uses  a  heavier 
warhead  than  in  previous  versions.  A  new  launcher  system  was  designed,  and 
the  weapon  system  is  able  to  carry  12  missiles,  whereas  previously  it  carried 

10  missiles.  Nine  countries  have  adopted  the  weapon  system.  The  target 
engagement  sequence  for  the  Paladin  is  flowcharted  in  Figure  5.  Characteris¬ 
tics  of  the  Paladin  are  identified  in  Table  1  from  a  description  of  the  Roland 

11  Mobile  Anti-Aircraft  Weapon  System.  The  French-German  Euromissile 
Consortium  developed  the  Roland  system. 
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Figure  2 


Air  Defense,  Anti-Tank  System  (ADATS)  target  engagement 
sequence  functions. 
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Table  1 


Weapon  System  Characteristics  (Jane’s  Weapon  Systems,  1986-87;  and 
Aviation  Week  &  Space  Technology,  1987) 


Air  Defense 

8  Ready- to-Flre  Missiles 

Anti-Tank 

Air  Targets 

System 

Anti-Tank  Defense 

(ADATS) 

Autonomous  Fire  Unit 

Propulsion  Solid,  Smokeless  Rocket  Motor 

Range  8+  km 

Search  Range  20  to 

1  Cannon  25  mn 

Machine  Gun  ,50  Caliber 

Bradley  Chassis 

Liberty 

6  Ready- to-Flre  Missiles 

(evolution 

(12  Missiles,  Production  Version) 

of  the 

40  Targets  an  Display 

Shahine) 

Threat  Evaluation  &  Tracking  18  Targets 

1  or  2  Acquisition  &  Coordination  Units,  &  2,  3,  or  4  Fire 
Units 

Acquisition  Range  19.5  to 

Tracking  Range  17  to 

Tracked  Vehicle  or  Heavy-Duty  Wheeled 

Reaction  Time  6  seconds 

AMX-30  Tank  Chassis 

(M1A1  Tank  Chassis  &  2  Cannons  25  am,  Production  Version) 

Tracked 

8  Ready- to-Flre  Missiles 

Rapier 

Air  Targets 

Anti-Tank  Defense 

Autonomous  Fire  Unit 

Propulsion  Integral 

2-Stage  Solid  Propellent  Motor 

Range  7  to 

Amphibious 

Cab  Design  for  NBC  Gear 

2  Machine  Guns  .50  Caliber 

EMC/RCM  748  Tracked  Vehicle 

(Bradley  Chassis,  1  Cannon  25  mn,  Production  Version) 

Paladin 

10  Ready-to-Flre  Missiles 

(renamed 

Air  Targets 

version  of 

Missile  Tracking  IR 

Roland  II) 

Autonomous  Fire  Unit 

Range  16  to 

M933  Tracked  Vehicle 

(M1A1  Tank  Chassis  &  1  Cannon  25  mn,  Production  Version) 

Figure  3 


Liberty  (evolution  of  the  Shahine)  target  engagement  sequence 
functions. 
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Figure  4.  Tracked  Rapier  target  engagement  sequence  functions 


Figure  5 


Paladin  (renamed  version  of  the  Roland  ZX)  target  engagement 
sequence  functions. 
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FAADS  TES  Specific  to  the  Sgt  York  Fire  Unit 

The  content  of  the  FAADS  data  base  described  In  this  report  came  from  the 
first  Follow-on  Evaluation  of  the  Sgt  York  system.  Therefore,  it  is  ap¬ 
propriate  at  this  time  to  look  at  the  procedures  for  conducting  the  target 
engagement  sequence  as  they  were  structured  for  the  Sgt  York  crews.  The 
amplification  of  procedures  will  be  important  to  understanding  and  interpret¬ 
ing  research  findings  regarding  crew  decisions,  crew  activities,  and  weapon 
system  processing.  The  concept  of  "system  modes"  is  key  and  pervasive  to 
learning  and  understanding  Sgt  York  target  engagement  procedures. 

System  Modes  of  Operation 

The  Sgt  York  gun  system  had  over  a  dozen  modes  of  system  operation. 

In  addition,  there  were  off-line  modes  to  prepare  for  operations.  Modes  of 
operation  were  identified  and  clustered  differently  in  different  documents, 
such  as  the  Operator's  Manual  (Crew)  Volumes  I-III,  and  the  1553  Serial  Data 
Bus. 

Mode  Performance  Data  Included  in  Data  Base.  During  FOE  I  Force-on- 
Force,  not  all  system  modes  of  operation  were  tested.  Human  and  system 
performance  data  (target  engagement  event  times)  was  collected  and  inserted 
into  the  FAADS  data  base  only  for  System  Acquisition  Modes  8,  9,  A,  B,  and  C. 
These  are  all  that  this  report  (e.g.,  the  data  base)  provides  data  for. 

(There  is  another  acquisition  mode,  mode  7,  for  which,  unfortunately,  no  data 
was  obtained.)  So,  in  the  most  important  respect,  having  entries  in  the 
performance  data  base,  only  these  acquisition  modes  are  addressed  by  this 
document. 

However,  to  provide  the  reader  with  some  idea  of  the  other  modes,  they  will 
also  be  listed  or  briefly  described  in  the  next  several  pages. 

Radar  Auto  mode.  Radar  Auto  mode  was  the  fastest  target  engagement 
method.  In  Radar  Auto,  the  system  pointed  to  a  top  priority  target  when  it 
was  within  a  predetermined  range.  Automatic  calculation  was  performed  for 
gun  lead  and  superelevation  angles.  The  system  was  directed  to  the  greatest 
threat  when  more  than  one  engageable  target  appeared.  When  the  RDR  switch  was 
set  to  AUTO,  the  PALM  SWITCH  was  enabled;  and  when  the  TURRET-GUN  switch  was 
set  to  ENABLE,  the  turret  could  slew  with  little  or  no  warning.  The  gunner 
watched  the  target  in  the  gunsight,  and  the  squad  leader  watched  the  display 
for  target  data  or  searched  for  additional  targets. 

Radar  Pointer  mode.  The  second  radar  method  used  to  acquire  targets  was 
termed  the  Radar  Pointer  mode.  The  operator  could  designate  with  a  cursor  a 
target  of  any  priority  during  Radar  Pointer  mode.  Only  one  target  could  be 
designated  with  the  pointer  symbol  on  the  display.  Once  the  target  had  been 
pointed,  the  target  was  watched  by  search  radar.  While  the  radar  was  tracking 
the  pointer-designated  target,  other  targets  could  also  be  engaged.  While  in 
the  Radar  Pointer  mode,  it  was  possible  to  acquire  targets  even  though  they 
may  have  been  beyond  the  recommended  firing  range.  Crew  members  could  acquire 
a  designated  target  under  Radar  Pointer  mode  at  any  time. 

Radar  Jammer  mode.  Radar  Jammer  mode  was  designed  to  acquire  on-board 
radar  jammers,  and  was  considered  a  variation  of  Radar  Pointer  method.  If  a 
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sector  was  affected  by  jamming,  the  radar  automatically  indicated  the  sector 
on  the  display.  The  display  indicated  the  calculated  center  of  the  jammed 
sector  (see  Figure  6).  Gunner  and  Squad  Leader  were  to  designate  jammer  cues 
as  though  they  were  targets. 

System  Laser.  The  laser  could  be  combined  with  any  radar  engagement 
method  (Radar  Auto,  Radar  Pointer,  Radar  Jammer),  or  it  could  be  used  alone 
for  the  optical/laser  mode.  A  more  accurate  fire  control  solution  was 
produced  when  the  laser  mode  was  combined  with  a  radar  mode  (see  Figure  7). 

Laser  with  any  radar  mode.  When  the  laser  mode  was  combined  with  a  radar 
mode,  laser  data  was  added  to  the  track  radar  data  to  establish  a  combined 
firing  solution.  If  the  crewmen  were  tracking  in  the  radar  mode,  they  were  to 
lase  when  the  target  could  be  seen  in  the  gunsight.  Actually,  the  determining 
factor  as  to  whether  radar  and  lase  modes  should  be  combined  was  contingent  on 
whether  there  was  additional  time  to  supplement  the  radar  solution  with 
greater  accuracy  to  the  radar-based  fire  control  solution.  If  a  firing 
solution  for  radar  only  was  blinking  instead  of  solid,  then  if  time  allowed, 
the  crew  could  lase.  Since  a  blinking  center  firing  cue  indicated  that  a 
firing  solution  had  been  achieved  yet  was  out  of  recommended  range,  use  of  the 

laser  could  promote  an  improved  fire  solution  and  hopefully  a  solid  firing  cue 

indicating  that  the  target  was  in  recommended  range. 

System  optical/ laser .  It  was  possible  to  use  the  laser  range-finder 
alone  during  optical/ laser  engagements.  The  laser  rangefinder  consisted  of  a 
transmitter,  receiver,  and  an  electronics  unit.  Lasing  was  the  only  way  to 
provide  target  range  and  speed  information  to  the  fire  control  computer  in  the 
optical  mode.  Lasing  a  target  in  the  optical  mode,  using  the  laser  ran¬ 
gefinder,  resulted  in  the  display  of  only  the  target  lased.  Ground  and  air 

were  the  two  submodes  of  optical/laser  operation.  These  modes  were  selected 
using  a  GROUND-AIR  switch  on  the  gunner's  control  panel.  Ground  targets  were 
displayed  with  a  ground  target  symbol  only.  Air  targets  were  differentially 
displayed  as  slow  or  fast  targets.  Slow  targets  were  represented  by  a 
helicopter  symbol,  and  fast  targets  were  represented  by  a  fixed-wing  aircraft 
symbol.  Targets  that  were  in  range  were  displayed  as  larger  symbols  than  for 
targets  out  of  range  (see  Figure  8). 

Point-detonating  ammunition  was  automatically  selected  in  the  laser 
ground  mode  unless  it  was  not  available  or  the  crew  member  overrode  the 
automatic  selection  of  ammunition.  The  narrow  laser  beam  (0.5  mils)  as¬ 
sociated  with  the  ground  mode  corresponded  to  the  gunsight  inner  reticle. 

Live  ammunition  was  not  used  during  the  Force-on-Force  trials. 

In  laser  air  mode,  ammunition  was  selected  automatically  by  the  fire 
control  computer  according  to  its  evaluation  of  the  target.  The  wider  laser 
beam  (3.0  mils)  used  during  air  mode  allowed  for  easier  tracking  of  a  dodging 
air  target.  The  air  mode  laser  beam  corresponded  to  the  coverage  of  the  peri¬ 
scope  inner  reticle  circle  and  the  gunsight  outer  reticle  circle. 

Laser  effectiveness  was  degraded  by  conditions  such  as  fog,  clouds,  and 
smoke.  In  addition,  since  laser  firing  generates  heat,  the  laser  would 
automatically  stop  firing  when  the  laser  transmitter  temperature  reached  a 
pre-set  level. 
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Figure  6 


Figure  7 


Display  indicates  jammed  sector  and  jamming  center  cue. 


DISPLAY 


Firing  solution  from  radar  only.  ■  B  H 

Firing  solution  from  later  only.  Ci  B  B 

Firing  solution  from  both  radar  and  laser,  fl  ■  B 


Firing  solution  cues  for  radar  and  laser. 
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TARGET  SYMBOLS 

DESCRIPTION 

ENGAGEABLE 

OUT  OF  RANGE 

❖ 

<s> 

Top  priority  fixed-wins  target. 

<S> 

<t> 

Second  priority  fixed-wing  target. 

O 

o 

Lower  than  second  priority  fixed-wing 
target 

<3> 

<e> 

Fixed  wing  target  deleted. 

<c> 

<5 

Top  priority  helicopter  target 

<p 

Second  priority  helicopter  target. 

o 

Lower  than  second  priority  helicopter 
target 

Helicopter  target  deleted. 

CZ3 

Ground  vehicle. 

Figure  8.  Target  symbols  for  fixed-wing,  helicopter,  and  ground  vehicle. 
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Gun  Slaving  Modes.  Particular  subaystem  failures  could  occur  during 
which  normal  fire  control  solutions  were  not  produced  by  the  fire  control 
computer.  Instead  the  data  were  entered  by  the  operator  who  was  moving  the 
gunsight.  In  the  slave  mode,  guns  were  pointed  where  the  gunsight  was  pointed 
(sights  were  scoped  to  gunline).  Gun  superelevation  and  lead  angle  were 
entered  by  the  operator.  The  fire  control  computer  could  not  make  automatic 
ammunition  selection  or  burst  length  in  the  slave  mode.  Therefore,  the 
operator  had  the  additional  task  of  selecting  both  ammunition  type  and  burst 
length. 

Operators  were  alerted  to  the  occurence  of  the  slave  mode  condition  by  a 
SYSTEMS  STATUS  (SST)  NO  GO  lamp  on  the  gunner's  control  panel  (see  Figure  9). 
Failures  of  the  attitude  reference  unit  (ARU)  or  the  fire  control  computer 
could  result  in  a  SST  message  which  specifies  slave  mode. 

System  Firing  on  the  Move.  System  firing  on  the  move  Included  the 
optical/ laser  system  mode  and  all  radar  system  modes.  All  laser  and  radar 
systems  were  stabilized  to  allow  for  engagements  while  traversing  terrain  or 
on  road  march  conditions.  However,  a  slight  loss  of  accuracy  would  be 
anticipated  when  firing  on  the  move  versus  firing  from  a  stationary  position. 

Battle  Quick  Start  procedures.  Fire  unit  preparation  required  preli- 
minary  procedures  to  be  conducted  prior  to  target  engagement.  Preliminary 
procedures  included,  but  were  not  limited  to,  system  operational  checks, 
magazine  loading,  calibration  firing,  pre-engagement  checks,  radar  prepara¬ 
tion,  setting  search-sector  limit,  setting  primary  target  line,  selecting 
radar  channel,  setting  symbols  switch,  antenna  deployment,  etc.  Battle  Quick 
Start  procedures  could  be  used  in  an  emergency  to  achieve  operational 
readiness  for  the  fire  unit  in  less  time.  However,  when  Battle  Quick  Start 
procedures  were  used,  there  was  some  degradation  of  system  capabilities  and 
performance . 

System  Mode  Radar  Silent.  During  Radar  Silent  mode,  power  was  supplied 
by  vehicle  batteries  only.  Silent  mode  was  used  for  periods  of  observation 
and  surveillance  or  after  loss  of  high  voltage  AC  power.  Since  radar  did  not 
operate  without  system  AC  power,  Radar  Silent  mode  consisted  of  a  limited 
version  of  the  optical/laser  engagement.  Two  12-volt  batteries  were  used  for 
silent  mode,  and  four  12-volt  batteries  were  held  in  reserve  to  start  the  fire 
unit.  Gun  firing  and  turret  movement  were  performed  manually.  Either 
gunsight  or  periscope  could  be  used  to  manually  aim  the  40-mm  guns.  There 
were  limitations  in  using  Radar  Silent  mode.  For  example,  the  attitude 
reference  unit  (ARU)  and  the  laser  were  cooled  by  AC  power.  The  ARU  and  the 
laser  could  have  led  to  automatic  shutdown  without  cooling. 

For  purposes  of  the  Force-on-Force  FOE  I,  modes  of  operation  were 
primarily  radar  pointer  or  radar  supported  by  laser.  Crew  members  usually 
maintained  control  over  the  turret  by  tracking  each  target  manually.  Using 
this  approach,  they  avoided  the  sudden  and  hazardous  slewing  of  the  turret 
that  could  occur  when  in  the  Radar  auto  mode. 
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GUNNER'S  CONTROL. PANEL 


Figure  9.  System  status  NO  GO  lamp  indicates  slave  mode. 
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System  Mode  Non-Operative.  In  addition  to  system  modes  of  operation, 
several  non-operative  modes  (off-line,  preparatory  modes)  were  available  to 
the  crew.  They  were  special  non-tactical  modes  which  crew  members  were  not  to 
select  when  involved  in  an  engagement  sequence.  The  seven  non-operative 
modes  were  to  be  used  for:  (1)  magazine  loading  and  down-loading,  (2)  checking 
and  setting  boresights,  (3)  preparatory  procedures,  (4)  calibration  firing, 

(5)  maintenance  operations,  (6)  proficiency  trainer,  and  (7)  fault  isolation. 

System  Modes  of  Operation  and  the  1553  Data  Bus.  The  1553  Serial  Data 
Bus  carried  system  event  data  between  the  Data  System  Controller,  the  Fire 
Control  Computer,  and  the  Radar  Computer.  Switch  actions  by  the  Sgt  York  crew 
were  recorded  on  the  1553  Data  Bus.  The  Data  Bus  recorded  to  the  nearest  one- 
thousandth  of  a  minute  when  a  fire  unit  entered  each  system  mode  and  the  onset 
time  of  various  activities  as  well  (see  Table  2). 

Non-Acquisition  Modes.  There  were  six  non-acquisition  modes.  The  non¬ 
acquisition  modes  were  to  be  used  during  activities  such  as  searching, 
detecting,  identifying  and  classifying,  acquiring,  tracking,  and  lasing 
targets.  However,  it  was  not  possible  to  engage  a  target  while  functioning  in 
any  of  the  six  non-acquisition  modes.  (The  label  of  Non-Engagement  Modes 
seems  more  appropriate.)  For  this  reason,  the  times  at  which  the  FU  entered  a 
non-acquisition  mode  were  not  extracted  for  entry  into  the  FAADS  Target 
Engagement  Sequence  data  base.  The  six  non-acquisition  modes  were  as  follows: 

1  -  Gunner  Slave,  Radar,  Optical 

2  -  Gunner  Slave,  Radar  Auto 

3  -  SL  Slave,  Radar,  Optical 

4  -  SL  Slave,  Radar  Auto 

5  -  Gunner  Free,  Radar,  Optical 

6  *  Gunner  Free,  Radar  Auto 

Because  they  involve  the  Radar  Auto  mode,  non-acquisition  modes  2,  4,  and 
6  are  at  a  higher  level  of  automation  than  modes  1,  3,  or  5.  Non-acquisition 
modes  1,  3,  and  5  all  included  the  optical  component  which  required  more  time 
to  engage  than  the  radar  auto  system  mode. 

Non-acquisition  modes  1  and  2  both  used  Gunner  Slave,  and  modes  3  and  4 
both  used  Squad  Leader  Slave  procedures.  The  four  non-acquisition  modes  using 
gunner  and  squad  leader  slave  are  not  to  be  associated  with  the  System  Mode 
Slave  where  subsystem  failure  precluded  a  normal  fire  control  solution. 

During  Gunner  Slave  (modes  1  and  2),  the  squad  leader  retained  control  of 
the  periscope  to  maintain  search  capability.  However,  since  the  gunner  was  in 
control,  the  gunner  maintained  control  of  guns  and  gunsight.  The  guns  moved 
to  gunsight  line  of  sight.  The  gunner's  triggers  were  enabled,  and  the  squad 
leader's  triggers  were  not  enabled.  The  Operator's  Manual  (Crew)  (Volume  I  of 
III,  Gun,  Air  Defense  Artillery,  Self-Propelled:  40  mm,  M247,  Sgt  York,  July 
1984)  defined  the  slave  mode  as  follows:  "To  lock  either  guns,  gunsight,  or 
periscope  to  certain  position  and  then  bringing  remaining  two  to  the  same 
azimuth  and  elevation.” 

In  Squad  Leader  Slave,  the  squad  leader  maintained  control  of  the  guns, 
turret,  and  gunsight.  The  guns  and  gunsight  both  moved  to  the  periscope  line 
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of  sight  (LOS).  The  gunner  had  no  grip  control.  The  squad  leader's  triggers 
were  enabled,  but  the  gunner's  triggers  were  not  enabled. 

As  seems  evident  from  the  name,  there  was  no  slaving  of  the  guns  to  the 
gunsight  during  the  Gunner  Free  modes  (5  and  6).  System  designers  provided 
an  abundance  of  system  modes.  The  reasons  for  distinguishing  between  these 
two  modes  and  for  prohibiting  firing  when  in  an  otherwise  aggressive  "non- 
acquisition”  mode  are  not  expressed  in  the  Operator's  Manual.  Provision  of 
redundancy  does  not  seem  a  sufficient  reason. 

Acquisition  Modes.  There  were  six  Acquisition  modes.  Any  of  them  could 
be  used  throughout  the  engagement  sequence.  The  1553  Data  Bus  indicated  the 
time  at  which  a  system  mode  was  changed  by  a  crew  member,  and  what  system  mode 
the  Fire  Unit  was  in  throughout  the  engagement  sequence.  The  six  acquisition 
modes  were  as  follows: 

7  «  Gunner/ SL,  Laser  Track,  Radar  Auto 

8  *  Gunner/ SL,  Radar  Auto 

9  *  Gunner/ SL,  Laser  Track,  Radar  Pointer 

A  *  Gunner/SL,  Radar  Pointer 

B  *  Gunner/SL,  Laser  Track,  Radar,  Optical 

C  *  Gunner/SL,  Radar,  Optical 

Modes  7  and  8  represented  the  highest  level  of  automation  during  a  target 
engagement  sequence,  since  they  both  used  a  form  of  Radar  Auto.  Modes  9  and  A 
represented  a  medium  level  of  automation  with  the  use  of  Radar  Pointer.  Modes 
B  and  C  were  indicative  of  the  lowest  level  of  automation  based  on  the 
functions  required  to  perform  an  optical  engagement.  In  the  optical  mode,  the 
squad  leader  detected  and  located  a  target  visually  by  either  using  the 
periscope  or  by  operating  with  head  out  of  the  turret.  In  the  optical  mode, 
all  targets  had  to  be  identified  visually  by  one  of  the  crew  members.  Mode  7 
was  not  employed  during  the  Force-on-Force  Target  Engagement  Sequences  that 
were  entered  into  the  Data  Base  created  by  this  research  effort. 

1553  Data  Bus.  The  1553  Data  Bus  documented  the  system  modes  that  were 
used  during  the  engagement  sequence.  Of  some  interest  was  the  sequence  of 
system  modes  used  during  each  engagement.  The  number  of  acquisition  modes 
selected,  the  number  of  non-acquisition  modes  selected,  and  the  mode  applying 
at  time  of  trigger  pull  were  retrieved  and  are  shown  in  Table  5.  Table  2 
depicts  the  chronology  of  a  portion  of  Trial  1035  as  printed  by  the  1553  Data 
Bus  listing.  The  listing  indicates  that  System  Mode  9  (Laser  Track,  Radar 
Pointer)  was  the  system  mode  of  operation  throughout  the  firing  sequence.  The 
Fs  running  down  the  F  column  indicate  continuous  firing  during  the  time 
sequence  represented  on  the  table.  The  sequence  depicted  has  a  total  duration 
of  3.93  seconds. 

For  a  better  understanding  of  the  relationship  between  the  system  modes 
and  the  1553  Data  Bus,  knowledge  of  the  procedures  required  to  initiate  target 
acquisition  and  engagement  sequence  using  radar,  laser,  or  both  would  be 
beneficial.  The  target  acquisition  and  engagement  sequence  for  Sgt  York 
consisted  of  eight  categories  as  shown  in  Figure  10. 

Searching  -  The  acquisition  and  engagement  process  was  initiated  by  having 
the  crew  search  for  targets.  Detecting  is  to  find  a  potential  target. 
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Table  2 

1553  Data  Bus,  Trial  1035 


TIME 

^ - - - t 


^SYSTEM  MODE. 


FIRING  ,  ACTIVITY 

V _ „ J  <■ - „ - ' 

'V 


-9  43.053 _ 221  -220  9-.1..C  L  R  P  4  S  J11L-E _ CL _ LASER  D3.  WTDE  BPlt 

9  43.075  221.242  9  A  G  L  R  P  4  S  B  B  F  R  FCC  CLAIMS  RADAR  LOCJf 

.9  ..42.123 _ 221. 2S3  9IGLRP  4SBRF _ R _ _ E.CC_rLAIKS_RAHAR_BRO!! 

9  43.143  221.310  9- A  G  L  R  P  4  S  BIB.F  GOHSIGHT  BQRESIGHT  HC 

9  43.2.48  221-515  Q-t  C  T.  R  P  4  S  5  R  P _ R_ _ E.CC_C  LAIK  S  _RADAR_LQCA 

9  43.393  221.560  9  A  G  L  .  P  4BBBF  R  RADAR  CUE  BLINKING 

9  43.621  221. 7B8  91  G  L  .  P  4  B  B:B  F _ R _ ES.C— CLAIMS  RADAR  RPflK 

9  43.666  221.833  9-1GL.P'  4BBBF  R  FCC  CLAIMS  RADAR  LOCK 

-Q  _4?t.7S7  _  _221 .924-  9  A  C  L  P  P  4  S  B _B_g _ JP _ RADAR_CDE -SOLID _ 

9  43.893  222.060  9-AGLRP  4  S  B  B  F  R  FCC  CLAIMS  RADAR  BROS 

9  44.030  222.197  9  1GLRP  4  S  B  B  F _ R _ F_C£_C  L  AIMS_R  AD  A  R_L  OCR 

9  44.166  222.333  9AGLRP  4SBBF  R  FCC  CLAIMS  RADAR  BROE 

9  44.212  222.379  9-AGLRP  4  S  B  B  _F _ R  FCC  CLAIMS  RADAR  LOCH 

9  44.256  222.423  9  A  G  L  R  P  4  S  B1 B  F  IFF  NOT  FRIEND,  TARG2 

9  44.302  222.469  _9  A  C  L  R  P  4  S  B  B  F  R  FCC  CLAIMS  PtDft,q_BRPF 

9  44.439  222.606  9-AGLRP  4SBBF  R  FCC  CLAIMS  RADAR  LOCE 

9  44.484  222.651  _9- A  G  L  R  P  4  S  B!B'F _ R_ _ EC{L_CLAIKS_RAHAR_BROE 

9  44.530  222.697  9  A  G  L  R  P  4  S  B  B  F  R  FCC  CLAIMS  RADAR  LOCI 

9  44.575  222.742 _ 9-ACLRP  4  S  B  B  F _ R. _ E  £&— CLAIiiS_R  AD  AR  BROE 

9  44.665  222.833  9-AGLRP  4  S  B!  B  F  R  FCC  CLAIMS  RADAR  LOCE 

.9.  ,4.4.712 . 222.  879 _ 9  1  G  L  R  P  4  S  B  B  F  R _ £CJLj:LAIKSjaA0AR_E2J}E 

9  44.757  222.924  9AGLRP  4SBBF  R  FCC  CLAIMS  RADAR  LOCE 

9.  .44. .8 £3 . 223«ftl5  .9:  A  GUP  4.B„B.JL_F _ R. _ RADAR._CCTJE_B.LINSXRS _ 

9  44-893  223.060  9  A  G  L  .  P  4  B  B  B  F  R  FCC  CLAIMS  RADAR  BRQI 

9  44.939  223.106  9  AGL.  P  4  B.B.B.F _ R  FCC  CLAIMS  RADAR  LOCI 

9-45.075  223.242  9-AGL.P  4BBBF  R  FCC  CLAIMS  RADAR  BROB 

9-45.121  223.288  9- A  G  L  .  P  4  B.  B:b.  P  R  FCC  CLAIMS  RADAR  LOCI 

9  45.211  223.378  9AGLRP  4SBBF  R  RADAR  COE  SOLID 

9  45.393  223.560  9-AGLRP  4  S  B.B  F  R  FCC  CLAIMS  RADAR  BRnu 

9-45.439-  223.606  9-ACLRP  4  S  B:B.F  R  FCC  CLAIMS  RADAR  LOCK 

9  45.575  223.742  9-AGLRP!  4SBBF  R  FCC  CLAIMS  RADAR  RPtlF. 

9  45.666  223.833  9-AGLRP  4SB.BF  R  FCC  CLAIMS  RADAR  LOCE 

9-45.757  223.924  9-AGLRP  4  S  B.B.F  R  FCC  CLAIHS  RADAR  BROB 

9  45.802  223.969  9-AGLRP.  4SBBF  R  FCC  CLAIMS  RADAR  LOCK 

9  45.848  224.015  9  A  G  L  R  P  4  S  _3  B  L ....  B _ FCC_CLAIMS  RADAR  BRO? 

9-45.984  224.151  9-AGLRP  4  S  BIB.F  R  FCC  CLAIHS  RADAR  LOCE 

9  46.029  224.196  9  A  C  L  R  P  4  S  B  B  E _ IFF.  KPT  FRIEND^  TARGE 

9  46.302  224.469  9AGL.RP  4SBBF  R  FCC  CLAIMS  RADAR  BROE 

9  46.343  224.515  _  9  -  A  G  L  R  P  _  _4_S_  jL_B_F _ B _ EGC_.CLAI.«S_ RAP_AR_U1C.I 

9  46.529  224.696  9-AGLRP  4SBBF  R  FCC  CLAIHS  RADAR  BROE 


3..A&151 _ 224 *.924 , 9  A  g.L.-RJE _ 4_SJL_B..F _ R. - F.CC  _CL  AIK S _R ADA RJUDlft* 

9  46.847  225.014  9-AGLRP  4  S  B  B  F  IFF  NOT  FRIEND,  TARGE 

9  46.893  225.060  9-AGLRP  4  S  B:  B  F  R  FCC  CLAIKS__RADAR  BR0{ 

9  46.964  225.151  9AGLRP  4SBBF  R  FCC  CLAIMS  RADAR  LOCI 
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Figure  10. 


Target  acquisition  and  engagement  sequence  for  Sgt  York 
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Identifying  is  to  determine  whether  the  target  is  a  friend  or  a  foe. 
Classifying  is  to  determine  what  type  of  vehicle  or  aircraft  the  foe  is;  and 
to  determine,  for  multiple  threats,  which  target  possesses  the  greatest  degree 
of  threat.  Acquiring  -  Acquiring  the  target  required  directing  the  tracking 
system  to  the  target.  In  the  case  of  the  Sgt  York,  the  tracking  system  could 
have  been  the  radar,  laser,  or  a  combination  of  both.  Lasing  -  Actually, 
lasing  was  a  subset  of  acquisition  and  tracking,  since  it  could  have  been  used 
optionally  in  either  of  these  categories.  During  tracking,  the  acquired 
target  was  followed  by  radar  and/or  laser.  Engaging  is  to  fire  the  system's 
guns  at  the  target.  Terminating  is  the  ending  of  the  engagement  sequence,  and 
this  could  happen  in  either  of  two  ways.  One,  the  crew  member  firing  could 
stop  pulling  the  trigger.  Two,  the  fire  control  system  could  terminate  the 
firing  even  though  the  crew  member  continued  to  pull  the  trigger.  This  was 
another  aspect  of  the  automatic  functioning  of  the  fire  units. 

For  an  overview  of  the  layout  within  the  crew  stations,  see  Figure  11. 

This  illustrates  the  location  of  the  gunner  and  squad  leader  control  panels, 
the  display,  and  the  control  grips. 

Radar  Auto.  In  the  Radar  Auto  system  mode,  either  the  squad  leader  or  the 
gunner  activated  the  palm  switch,  and  the  turret  gun  switch  was  set  to 
enable.  The  RDR  switch  was  set  to  AUTO.  The  fire  control  computer  directed 
the  system  to  the  target  when  the  target  was  in  the  recommended  range.  The 
system  was  directed  to  the  target  of  highest  priority,  e.g.,  greatest  threat. 

A  solid  radar  cue  meant  the  target  had  been  acquired  by  the  radar.  In  the 
Radar  Auto  mode,  the  crew  decision  time  was  reduced  more  than  in  the  other 
system  modes.  In  the  Radar  Auto  mode,  once  the  auto  mode  had  been  set  up  by 
the  crew,  the  only  decision  for  the  crew  member  to  make  was  to  pull  the 
trigger  when  there  was  a  solid  fire  cue.  The  Radar  Auto  procedures  are 
illustrated  in  Figure  12. 

The  squad  leader  and  the  gunner  each  had  identical  right  control  grips. 

The  right  control  grip  had  a  toggle  switch  which  placed  the  system  in  Radar 
Auto  or  Radar  Pointer  acquisition  mode  (see  Figure  13). 

Radar  Pointer.  Using  the  Radar  Pointer  acquisition  mode,  it  was  possible 
to  designate  one  target  with  the  pointer  symbol  on  the  display.  Once  the 
target  was  pointed,  it  was  watched  by  search  radar.  It  was  possible  to 
engage  other  targets  while  the  radar  watched  the  designated  target.  The 
squad  leader  or  the  gunner  could  designate  the  target  using  the  Radar  Pointer 
method.  Radar  pointer  designation  was  performed  by  selecting  a  target  on  the 
display  and  holding  the  pointer  on  button  down.  At  the  same  time,  the  thumb 
tracker  was  used  to  direct  the  pointer  symbol  to  the  designated  target.  When 
the  pointer  on  symbol  touched  the  target  symbol,  the  POINTER  ON  button  was 
released.  The  pointer  locked  on  the  target  and  moved  with  the  target  on  the 
display.  The  flow  of  the  sequence  for  the  radar  pointer  activity  is  shown  in 
Figure  14. 

As  in  the  Radar  Auto  acquisition  mode,  either  the  squad  leader  or  the 
gunner  used  his  right  control  grip  for  radar  pointing.  In  this  procedure,  the 
toggle  switch  was  moved  to  the  pointer  position.  The  left  control  grip  was 
used  to  push  and  hold  down  the  POINTER  ON  button.  Concurrently,  the  THUMB 
TRACKER  was  used  to  direct  the  pointer  to  the  target  symbol  on  the  display 
(see  Figures  15  and  16). 
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!♦  GUNNER'S  CONTROL  PANEL:  Contains  controls  for  gun  and  ammunition  setup,  laser 
mode  selection  and  arming,  system  mode  selection,  and  16-button  keyboard  for  entering 
commands  and  data.  Also  contains  indicator  lamps  to  show  status  of  systems. 

2.  DISPLAY:  Shows  target  situation,  vehicle  coordinates,  system  status,  operating  menus, 
and  fault  isolation  data,  depending  upon  mode  selected. 

3  •  SQUAD  LEADER'S  CONTROL  PANEL:  Contains  controls  for  selecting  radar  mode 
and  search  sector,  arming  guns,  adjusting  status  of  display  symbols,  and  selecting 
power  on-line  status  of  major  systems.  Also  contains  indicator  lamps  to  show  status 
of  different  systems. 

4 .  POWER  DISTRIBUTION  UNIT  (PDU):  Contains  circuit  breakers  for  electrical  circuits  in 
turret  systems.  Includes  BATTLE  SHORT  SI  and  TUR  INTLK  OVERRIDE  S2  switches. 

5.  CONTROL  GRIPS:  Control  turret  and  gun  movement  and  target  engagement  functions. 


Figure  11.  Crew  compartment  components. 
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Figure  12.  Radar  Auto  procedures. 
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RADAR  AUTO 


Figure  13.  Right  control  grip,  Radar  Auto. 
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Figure  14.  Radar  Pointer  procedures. 


THUMB  THACKER 


Figure  15.  Right  control  grip,  radar  pointer  and  thumb  tracker 


Figure  16.  Left  control  grip. 
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Lasing  Targets.  There  were  Cwo  primary  modes  used  for  lasing  a  target. 
There  was  the  Optical/Laser  acquisition  mode  previously  described  as  engaging 
targets  without  the  support  of  radar  capability.  In  the  Optical/Laser  mode, 
the  IFF  did  not  identify  targets  so  that  targets  had  to  be  visually  identified 
during  tracking.  In  the  Optical  mode,  the  only  way  to  give  target  range  and 
speed  data  to  the  fire  control  computer  (FCC)  was  by  lasing. 

When  added  to  either  the  Radar  Auto  mode  or  the  Radar  Pointer  mode,  lasing 
a  target  could  provide  greater  accuracy  to  the  radar-based  fire  control 
solution.  Provided  that  the  target  could  be  seen  in  the  gunsight  and  that 
there  was  time  to  arm  the  laser,  greater  accuracy  was  possible.  A  flow 
diagram  of  the  lase  procedure  is  presented  in  Figure  17. 

To  initiate  the  lase  procedure,  the  GROUND-AIR  switch  was  set  for  target 
type  by  the  gunner.  The  switch  was  located  on  the  gunner's  control  panel.  In 
ground  mode,  the  narrow  laser  beam  matched  coverage  of  the  gunsight  inner 
reticle  circle,  and  point  detonating  ammunition  was  automatically  selected. 
After  the  GROUND-AIR  switch  had  been  set,  the  laser  switch  was  set  to  ARM  by 
the  gunner.  The  ARMED  Indicator  lamp  indicated  the  laser  armed  condition. 

See  Figure  18  depicting  the  gunner's  control  panel  for  laser. 

If  the  crew  decided  to  lase  a  target  without  radar,  then  the  gunner  was  to 
set  the  OPT  switch  to  slave  and  release.  This  would  place  the  gunner  in 
gunner  slave  condition.  The  gunner  was  then  in  control.  The  guns  moved  to 
gunsight  LOS,  and  the  gunner's  trigger  was  enabled.  Concurrently,  the  squad 
leader  was  free  to  move  the  periscope  for  search  activities,  and  the  squad 
leader's  trigger  was  not  enabled.  The  squad  leader's  OPT  switch  was  set  to 
free,  while  the  gunner's  switch  was  set  to  slave.  See  Figure  19  for  a  line 
drawing  of  the  gunner's  right  control  grip  which  indicates  the  OPT  toggle 
8 witch. 

After  the  laser  had  been  armed,  the  laser  track  button  was  pushed. 

Pushing  the  laser  track  button  erased  any  earlier  target  range  data.  Gunsight 
control  was  given  to  the  gunner,  and  the  laser  was  prepared  for  firing. 

Figure  20  represents  the  gunner's  left  control  grip  and  the  location  of  the 
laser  track  button. 

The  thumb  tracker  (right  control  grip)  was  used  to  move  the  center  of  the 
small  reticle  circle  to  the  selected  target  area  (see  Figure  19).  As  soon  as 
the  circle  was  moving  at  the  same  rate  as  the  lased  spot,  the  laser  track 
button  was  released.  There  were  laser  cues  on  the  display.  The  thumb  tracker 
was  used  to  continue  to  make  small  corrections  as  the  target  was  tracked. 
Periscope  and  gunsight  eye  pieces  blinked  until  the  range  data  was  received. 
When  the  range  data  was  accepted,  the  laser  cue  came  on  steady.  The  target 
would  now  be  considered  in  laser  acquisition,  and  could  be  engaged. 
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Figure  17.  Lase  procedures. 
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A  TES  Performance  Data  Base 


During  the  first  half  of  1985,  follow-on  evaluation  tests  (FOE  I)  were 
conducted  to  evaluate  the  performance  of  the  Sgt  York  Air  Defense  Gun.  FOE  1 
for  Sgt  York  had  two  phases,  Force-on-Force  and  Live  Fire.  The  Force-on- 
Force  phase  included  29  test  trials  in  which  Sgt  York  fire  units  were  involved 
and  which  met  pre-established  criteria  for  validity.  As  described  in  the 
preceding  section,  a  1553  Data  Bus  was  connected  to  each  Sgt  York  fire  unit 
taking  part  in  the  Force-on-Force  trials.  Consequently,  it  was  possible  to 
monitor  the  actions  of  the  crew  members  as  indicated  by  their  button-pushing 
and  switch  actions.  For  example,  records  of  switch  actions  showed  when  the 
radar  pointer  control  was  pressed  or  released,  when  the  laser  was  turned  on 
and  off,  and  when  the  trigger  was  engaged  and  released.  From  the  printout  of 
the  data  bus  Information  (See  sample  in  Table  2),  it  was  possible  to  tell  what 
was  done,  when  it  was  done,  and  by  whose  controls,  the  squad  leader's  or  the 
gunner's. 

By  extracting  data  from  the  data  bus  record  of  activity  during  the  Force- 
on-Force  trials,  an  extensive  body  of  data  was  compiled  on  the  target 
engagement  sequences  that  occured  during  Force-on-Force  test  trials.  A  total 
of  15  complete  trials  was  analyzed.  In  selecting  the  trials  to  be  analyzed, 
trial  variables  were  considered  and  trials  chosen  so  that  all  test  conditions 
were  represented  within  the  selected  trials.  To  make  the  effort  manageable, 
while  maximizing  the  usefulness  of  the  obtained  data  distributions,  only 
actions  which  led  to  complete  target  engagement  sequences  were  extracted. 

That  is,  for  an  action  or  series  of  actions  to  appear  in  the  data  base  that  is 
presented  and  discussed  in  this  report,  it  had  to  culminate  in  firing  on  a 
target.  Each  target  that  was  actually  fired  on  during  the  15  trials  selected 
for  analysis  was  tracked  back  to  its  initial  appearance.  Action  pertaining  to 
that  target  then  was  identified  and  recorded  for  the  period  from  time  of 
appearance  (the  start  of  an  engagement)  to  the  time  that  firing  on  that  target 
ceased  (the  end  of  an  engagement).  A  total  of  271  such  target  engagements 
were  Identified  and  analyzed  for  this  report. 

From  start  to  end  of  a  target  engagement  sequence,  a  number  of  tasks  had 
to  be  carried  out.  The  target  had  to  be  identified,  selected,  and  classified; 
it  had  to  be  tracked  and  a  Fire  Solution  calculated  to  make  interception 
possible.  A  firing  decision  had  to  be  made  and  implemented.  The  fire  units' 
computers  were  available  to  aid  the  operators  throughout  the  target  engagement 
sequence.  As  has  been  explained  in  the  preceding  section,  several  operational 
modes  were  available  to  the  Sgt  York  crewmen  so  that  they  could  choose 
appropriate  levels  of  automation. 

The  conditions  and  parameters  of  the  Sgt  York  FOE  I  tests  are  described 
briefly  below  to  provide  a  context  for  later  discussions  of  variables  used  in 
the  categorization  and  presentation  of  data.  Following  the  test  description, 
the  final  and  major  portion  of  this  chapter  presents  the  actual  data  obtained 
for  the  FOE  1  tests. 

The  Sgt  York  FOE  I  Tests 

During  the  Sgt  York  FOE  I  Force-on-Force  trials  which  were  conducted  at 
the  Combat  Development  Experimentation  Center  (CDEC)  at  Fort  Hunter-Liggett, 
CA,  there  were  a  total  of  52  valid  trials  conducted.  Of  this  total,  12  were 
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Vulcan  baseline  trials,  11  others  were  combined  Chaparral  and  Vulcan  trials, 
and  29  included  Sgt  York  fire  units.  It  was  from  this  group  of  29  trials  that 
the  subset  of  15  was  selected  for  analysis. 

The  Blue  Force  portrayed  the  friendly  force  and  the  Red  Force  simulated 
the  enemy.  In  an  attempt  to  have  test  conditions  represent  the  variety  of 
situations  that  might  be  encountered  in  an  actual  battlefield,  the  Blue 
battalion  task  force  was  assigned  to  a  series  of  scenarios  to  be  conducted 
against  enemy  vehicles  and  aircraft  under  various  electronic  warfare  condi¬ 
tions  with  simulated  enemy  aircraft  employing  a  variety  of  tactics,  and  with 
crews  wearing  nuclear,  biological,  and  chemical  (NBC)  gear  (i.e.,  Mission- 
Oriented  Protective  Posture  (MOPP)). 

During  each  of  the  Sgt  York  trials,  from  three  to  five  Stinger  crews  were 
present  for  the  Blue  Force.  In  addition,  two  Chaparral  fire  units  were 
deployed  to  the  rear  of  the  battalion  task  force.  The  Blue  Force  also  had 
from  21  to  26  M-l  tanks,  from  10  to  13  M-3s  (infantry  fighting  vehicles),  and 
usually  an  AH-1  (attack  helicopter),  although  on  Trial  1026  an  OH-58  (observa¬ 
tion  helicopter)  was  used  as  a  substitute. 

The  Red  Force  included  two  A-lOs  (fixed-wing  aircraft)  representing 
Frogfoot  aircraft,  two  A-7s  (fixed-wing  aircraft)  representing  FITTERS,  and 
from  two  to  four  AH-64s  (attack  helicopters)  acting  as  HAVOC  or  HIND  heli¬ 
copters  depending  on  the  particular  test  conditions  to  be  met.  In  addition, 
there  were  other  support  elements,  including  at  least  16  M-60  tanks  represent¬ 
ing  T-80  tanks. 

Each  Sgt  York  FOE  I  trial  had  originally  been  planned  to  have  a  duration 
of  72  hours.  However,  due  to  the  constraints  of  instrumentation,  data 
collection  for  each  trial  was  limited  to  approximately  20  to  30  minutes. 

Even  so,  the  Sgt  York  crew  members  were  in  their  crew  compartments  for  at 
least  2  to  3  hours  for  each  trial. 

Each  trial  was  assigned  a  sequential  trial  number  and  each  trial  was 
assigned  to  one  of  three  scenarios,  to  one  of  three  helicopter  tactical 
conditions,  to  one  of  three  ECM  levels,  to  day  or  night  time,  and  to  one  of 
two  MOPP  levels.  These  conditions  are  described  briefly  below. 

Scenario  Conditions.  Three  different  scenarios  were  used  for  the  Force- 
on-Force  trials:  (1)  Delay,  (2)  Attack,  and  (3)  Road  March.  Each  scenario 
differed  by  the  technical  mission  assigned  to  the  Blue  Force.  Two  versions  of 
each  scenario  type  were  implemented.  The  chief  difference  between  the  two 
versions  was  in  the  direction  of  orientation  and  thus  the  specifics  of  the 
terrain  conditions  encountered. 

Valid  Sgt  York  trials  included  10  Delay  scenario  trials,  14  Attack 
scenario  trials,  and  5  Road  March  scenario  trials.  Four  Delay  scenario 
descriptions  used  site  1,  Gabilan  Valley,  southeast,  and  six  Delay  scenarios 
used  site  2,  San  Miguellto  Loop,  northwest.  Six  Attack  trials  used  the  San 
Mlguelito  Loop  terrain  site,  southwest,  designated  as  site  3.  An  additional 
eight  Attack  scenario  descriptions  used  terrain  site  4,  Gabilan  Valley, 
northwest.  Only  one  Road  March  scenario  trial  used  terrain  site  5,  San 
Mlguelito  Loop,  southeast.  Road  March  terrain  site  6,  Gabilan  Valley,  was 
used  in  four  different  trials. 


Helicopter  Tactics.  Another  of  the  variables  Introduced  into  the  Sgt 
York  FOE  I  trials  concerned  the  tactics  used  by  the  simulated  enemy  rotary- 
wing  aircraft.  Two  different  types  of  threat  were  represented.  Havoc  and 
Hind  aircraft.  That  is,  in  a  given  trial,  threat  rotary-wing  aircraft 
represented  either  the  Hind  or  the  Havoc. 

Two  tactical  variations  were  introduced  when  the  Hind  was  the  helicopter 
simulated.  In  some  trials,  the  Hind  used  the  tactic  of  hovering,  and  in  other 
trials  it  used  the  tactic  of  running.  The  simulations  of  the  Havoc  only  used 
the  tactic  of  hovering.  Thus,  there  were  three  combinations  of  helicopter- 
plus-tactical  variations,  one  of  which  was  assigned  to  each  trial.  Each  trial 
was  either  a  Havoc/hover,  a  Hind/hover,  or  a  Hind/running  trial. 

ECM  Conditions.  The  countermeasure  types  were  termed  Benign,  Design,  and 
Initial  Operational  Capability  (IOC).  In  those  trials  assigned  to  the  Benign 
ECM  conditions,  no  electronic  countermeasures  were  introduced.  The  Design 
condition  was  defined  as  electronic  countermeasures  consistent  with  Sgt  York 
design  specifications;  it  was  the  level  originally  intended  to  be  used  in  Sgt 
York  trials.  The  Initial  Operating  Capability  (IOC)  condition  represented  an 
advanced  ECM  environment,  consistent  with  the  level  of  electronic  counter¬ 
measures  that  Sgt  York  was  expected  to  encounter  when  fielded.  Thus,  it  was 
the  most  stringent  ECM  condition  used  during  Sgt  York  FOE  I  trials. 

Time-of-Day.  FOE  I  was  initially  seen  as  an  opportunity  to  evaluate  the 
ability  of  the  Sgt  York  to  function  under  both  day  and  night  conditions.  Both 
road  march  scenarios  were  scheduled  to  be  run  at  night  and  in  the  day,  but 
operational  difficulties  and  safety  hazards  encountered  in  the  first  nighttime 
trial  (Trial  1040,  Road  March  6)  led  to  a  decision  prior  to  the  second  night 
road  march  to  deploy  the  fire  units  along  the  route  of  march  during  the  pre¬ 
dusk  hours  while  light  was  still  available  so  that  they  did  not  have  to  be 
repositioned  in  the  dark.  Thus,  the  second  night  road  march  was  only 
nominally  a  night  trial.  Because  the  operational  conditions  of  the  Road  March 
5  trial  differed  from  those  of  the  earlier  night  road  march,  only  the  early 
trial  was  used  in  the  Day-Night  comparison. 

MOPP  Level.  Two  levels  of  MOPP  equipment  were  introduced  as  a  variable 
in  the  FOE  I  trials.  Two  trials  were  conducted  with  the  crews  wearing  MOPP 
Level  4.  All  other  Force-on- Force  trials  were  conducted  at  MOPP  Level  1.  The 
crew  members  started  the  trial  by  wearing  MOPP  gear  around  their  lower  legs 
with  overboots  on.  The  tactical  operating  procedure  was  to  suit  up  partially 
before  entering  a  vehicle.  Crew  members  wore  the  MOPP  overgarments  around 
their  lower  legs  for  Level  1  MOPP.  For  the  two  MOPP  4  trials,  the  MOPP 
overgarments  were  pulled  up  and  masks/hoods  and  gloves  were  donned  (Babbitt, 
1987). 

The  Sgt  York  FOE  I  Data  Base 


This  section  presents  the  data  base  compiled  by  extracting  data  from  the 
Sgt  York  FOE  I  Force-on-Force  test  trials.  First,  Table  3  presents  the 
performance  data  base,  showing  trial  by  trial  mean  times  for  each  of  11 
segments  of  the  sequence.  Table  4  summarizes  the  mean  times,  segment  by 
segment,  and  Includes  the  standard  deviation  and  N  for  each  measure.  Table  5 
presents  data  on  the  number  of  targets  displayed  when  the  selected  target 
first  appeared,  the  range  at  which  the  selected  target  first  appeared,  and 
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information  on  system  modes  applicable  to  each  engagement.  It  is  organized, 
as  was  Table  3,  by  trial  number  and  by  fire  unit  and  engagement  number  within 
each  trial.  The  elements  of  Table  3  are  discussed  below  in  detail. 


Table  3  presents  the  data  extracted  from  the  records  of  the  actual 
performance  of  crew  members  and  fire  units  as  they  took,  part  in  the  Sgt  York 
FOE  I  Force-on-Force  test.  As  noted  earlier,  15  of  the  29  valid  Sgt  York 
trials  were  the  focus  of  detailed  analysis.  All  completed  target  engagement 
sequences  (i.e.,  those  in  which  a  target  was  actually  fired  on)  in  those  15 
trials,  a  total  of  271  target  engagement  sequences,  were  analyzed.  In  Table 
3,  data  on  those  271  engagements  are  ordered  by  trial  number,  the  first  trial 
presented  being  Trial  1020  and  the  last  trial  presented  being  Trial  1048.  A 
new  page  is  started  with  each  change  of  trial  number,  and  some  trials  run  over 
a  single  page  of  description. 

Within  each  trial,  the  data  are  ordered  by  fire  unit  number,  from  1  to  5, 
and  within  fire  unit  by  engagement  number.  Fire  unit  number  and  engagement 
number  are  shown  in  the  first  column  of  Table  3.  If  a  fire  unit  number  is 
missing  from  the  record  of  a  given  trial,  it  Indicates  that  the  fire  unit 
either  did  not  take  part  in  that  trial  or  did  not  finish  any  complete  target 
engagements  during  that  trial. 

For  each  engagement  detailed  in  Table  3,  times  are  shown  for  11  different 
time  intervals.  These  11  intervals  represent  four  separate  classes  of  data: 
Crew  decision  times,  crew  action  times,  system  processing  times,  and  summary 
measures.  These  classes  are  named  in  the  top  line  of  Table  3.  In  the  second 
line,  the  column  headings  name  the  11  time  intervals.  These  11  intervals, 
which  are  dependent  variables,  are  defined  and  described  below.  In  each  case, 
the  event  or  action  that  began  the  interval  and  the  one  that  ended  It  are 
identified. 

Select/Classify  Target.  The  selection  interval  began  when  a  target 
appeared  on  the  plasma  display.  This  event  was  indicated  on  a  plot  of  the 
targets  displayed  that  was  taken  from  the  1553  data  bus.  It  showed  targets 
by  number  or  letter  with  range  (0-10  km)  shown  across  the  top  of  the  plot 
and  time  along  the  left  margin.  The  selection  interval  ended  when  the 
operator  (squad  leader  or  gunner)  depressed  either  the  radar  pointer  or  the 
laser  switch  to  begin  target  acquisition.  The  mean  time  taken  to  select  a 
target  was  7.9  seconds,  for  221  engagements. 

Point  Target.  This  column  records  the  time  it  took  a  crew  member  to  get 
a  target  pointed;  I.e.,  the  interval  between  pointer  depressed  and  pointer 
release,  so  long  as  the  release  was  followed  by  the  appearance  on  the  1553 
data  printout  of  a  target  number  under  "Target  Pointed."  To  point  a  target,  a 
crew  member  had  to  track  the  target  while  holding  the  pointer  switch 
depressed.  The  mean  time  it  took  to  point  a  target  was  2.3  seconds,  for  155 
engagements. 

Lase  Target.  This  column  records  the  time  it  took  a  crew  member  to 
indicate  a  target  by  lasing  it.  The  time  entered  represents  the  interval 
between  laser  depression  and  laser  release,  if  that  release  was  followed  by 
the  appearance  on  the  1553  data  printout  of  a  target  number  under  "Target 
Engaged."  The  mean  time  that  the  laser  switch  was  depressed  was  1.1  seconds, 
for  66  engagements. 


Acquisition  of  Target.  Once  the  tracking  radar  had  locked  onto  the 
target  selected  and  was  able  to  follow  it,  the  Radar  Pointer  Button  or  the 
Laser  Button  was  released  by  computer  signal.  Later,  the  1553  data  bus 
Indicated  that  a  radar  cue  or  a  laser  cue  was  on.  The  interval  between  the 
time  that  the  crew  action  (pointing  or  lasing)  terminated  and  the  time  that 
the  radar  or  laser  cue  came  on  was  recorded  as  the  acquisition  Interval.  The 
mean  acquisition  time  for  the  221  engagements  involving  Point  Target  and  Lase 
Target  was  3.5  seconds.  The  mean  acquisition  time  for  the  271  engagements 
recorded  in  Table  3  was  4.2  seconds. 

Fire  Solution  by  Computer:  Blinking  Cue.  Once  the  target  had  been 
acquired  by  the  tracking  radar  and  had  range  as  well  as  elevation  and  azimuth 
information,  the  fire  control  computer  was  able  to  calculate  a  fire  control 
solution.  When  that  solution  had  been  obtained,  the  firing  cue  came  on  in  the 
Sgt  York  crew  compartment.  A  firing  cue  light  might  blink  on  and  off,  or  it 
might  simply  come  on  and  stay  on.  The  former  case  was  referred  to  as  a 
“blinking"  firing  cue  and  the  latter  case  was  referred  to  as  a  "solid"  firing 
cue.  A  blinking  cue  indicated  a  less  certain  solution  than  did  a  solid  cue. 
The  Interval  between  the  appearance  of  the  radar  or  laser  cue  and  the 
subsequent  appearance  of  the  blinking  firing  cue  was  entered  as  the  time  it 
took  for  Fire  Solution  by  Computer:  Blinking  Cue.  Whether  or  not  to  fire  on 
a  blinking  firing  cue  rather  than  waiting  for  a  solid  cue  (that  might  or  might 
not  appear)  was  left  to  the  discretion  of  the  operator.  If  the  firing  cue 
light  stopped  blinking  and  became  a  solid  firing  cue  before  the  trigger  was 
activated,  ”NA"  was  entered  in  the  blinking  cue  column  for  that  engagement. 

On  65  of  the  271  engagements,  firing  began  on  a  blinking  fire  cue.  For  those 
65  cases,  the  mean  time  taken  by  the  computer  to  produce  a  firing  solution  was 
4.0  seconds. 

Fire  Solution  by  Computer:  Solid  Cue.  If  the  firing  cue  light  came  on 
and  stayed  on,  the  time  between  Target  Acquisition  (indicated  by  the  ap¬ 
pearance  of  a  radar  cue  or  a  laser  cue  light)  and  the  onset  of  the  solid 
firing  cue  was  entered  as  Fire  Solution  by  Computer:  Solid  Cue.  A  blinking 
fire  cue  might  or  might  not  precede  a  solid  fire  cue,  but  the  time  entered 
under  "solid  cue"  was  always  calculated  from  the  end  of  acquisition.  Thus, 
the  time  to  obtain  a  solid  firing  solution  was  not  affected  by  whether  or  not 
a  blinking  fire  cue  occurred,  and  the  time  to  solid  firing  cue  could  be 
compared  across  engagements.  If  a  solid  firing  cue  came  on  after  a  blinking 
firing  cue  and  after  firing  had  already  begun,  the  time  to  onset  of  a  solid 
cue  is  shown  in  parenthesis  in  Table  3  and  is  not  taken  into  account  in 
calculating  the  mean  time.  If  the  firing  cue  did  not  stop  blinking,  ”NA"  was 
entered  under  "Solid  Cue."  For  the  206  engagements  for  which  there  was  a 
solid  fire  cue  presented  prior  to  trigger  depression,  the  mean  time  to  onset 
of  the  solid  fire  cue  was  2.6  seconds. 

Comparing  the  mean  time  for  Fire  Solution:  Blinking  (4.0  seconds)  with 
mean  time  for  Fire  Solution:  Solid  (2.6  seconds)  suggests  that  the  crew 
members  may  have  had  some  informal  criterion  for  how  long  they  should  wait  for 
a  fire  cue  to  become  solid,  and,  if  the  fire  cue  continued  to  blink  after  that 
time  Interval  had  passed,  the  operator  decided  not  to  wait  longer  but  to  fire 
on  the  blinking  cue.  Had  he  waited,  perhaps  there  would  have  been  a  time  for 
"fire  cue  solid"  instead  of  "NA"  in  that  column.  It  appears  that  it  takes 
less  time  to  achieve  a  solid  fire  cue  than  to  achieve  a  blinking  fire  cue,  it 
may  be  that  for  some  engagements  the  solution  was  more  difficult  or  tenuous 
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and  on  such  engagements  the  solution  not  only  took  longer  but  only  reached  the 
“blinking  cue"  stage. 

Decision  to  Fire;  After  Blinking  Cue.  The  time  interval  between  the 
appearance  of  a  firing  cue  and  trigger  depression  by  a  crew  member  was 
recorded  as  decision-to-fire  time.  As  noted  earlier,  although  a  solid  firing 
cue  indicated  a  higher  probability  of  a  successful  engagement,  a  crew  member 
could  decide  to  fire  on  a  blinking  cue.  Firing  began  a  mean  of  1.7  seconds 
following  the  onset  of  a  blinking  fire  cue  and  1.3  seconds  after  a  solid  fire 
cue.  If  there  is  an  entry  in  the  column  headed  “After  Blinking  Cue,"  it 
indicates  that  on  that  engagement,  the  operator  fired  on  a  blinking  cue.  The 
number  entered  indicates  the  delay  in  seconds  from  time  of  onset  of  blinking 
fire  cue  to  time  of  trigger  activation.  If  the  trigger  was  not  depressed 
while  the  fire  cue  was  blinking,  “NA"  is  entered. 

Decision  to  Fire;  After  Solid  Cue.  The  time  between  onset  of  a  solid 
fire  cue  and  trigger  depression  is  entered  in  this  column.  If  the  operator 
fired  before  the  solid  firing  cue  appeared,  “NA"  is  entered.  On  one  en¬ 
gagement  (Trial  1046,  Fire  Unit  2,  Engagement  6),  a  negative  number,  -0.1,  is 
entered  under  "After  Solid  Cue."  On  this  occasion,  the  gunner  activated  the 
trigger  just  prior  to  the  appearance  of  the  firing  cue.  In  this  case  when  the 
fire  cue  came  on,  it  was  solid  from  the  onset. 

Time  to  Fire.  This  column  represents  a  summary  of  the  preceding  steps  of 
the  TES.  It  tells  how  long  it  took  after  target  presentation  for  firing  to 
begin.  The  overall  mean  Time  to  Fire  for  all  271  engagements  was  16.5 
seconds,  except  for  times  enclosed  in  parentheses;  those  steps  are  not 
included  in  the  Time-to-Fire  total  because  they  occurred  simultaneously  with 
other  steps. 

Fire  Duration.  This  column  indicates  how  long  firing  continued  for  each 
engagement.  That  is,  it  records  the  time  interval  from  the  beginning  of 
trigger  pull  to  the  end  of  the  engagement.  Some  engagements  were  ended  when 
the  gunner  released  the  trigger.  In  other  instances,  the  weapon  system  ended 
the  engagement  (stopped  firing)  when  radar  lock-on  was  lost.  According  to  Sgt 
York  operating  manuals,  firing  was  to  continue  for  3  to  7  seconds  when  firing 
in  burst  select  mode.  The  mean  Fire  Duration  for  the  271  engagements  was  8.0 
seconds. 

Total  Engagement  Duration.  This  column  summarizes  the  time  interval  from 
the  appearance  of  the  target  to  the  cessation  of  firing.  The  mean  Engagement 
Duration  across  all  271  trials  was  24.6  seconds. 
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Performance  Data  Base.  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1020  (4  Fire  Units)  32  Engagements) 
(All  times  in  seconds.) 
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Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1020  (4  Fire  Units;  32 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

•*No  crew  action  indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions. 


Performance  Data  Base  (Cont'd. ) .  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1021  (3  Fire  Units;  14 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

••No  crew  action  indicated  in  1553  data  base. 

(  )  Action  occurred  simultaneously  with  other  events/functions 


Performance  Data  Base  (Cont’d.).  Target  engagement  time  durations  for  Sgt  York  FOB  I  Trial  1022  (4  Fire  Units*  34 
Engagements).  (Ml  times  in  seconds.) 
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♦Crew  action  was  first  indicator  of  target  appearance. 

•  •No  crew  action  indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOB  I  Trial  1022  (4  Fire  Units;  34 
Engagements).  (All  times  in  seconds.) 
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Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  X  Trial  1023  (3  Fire  Unitsj  13 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

••No  crew  action  Indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions. 


I 


I 


♦Crew  action  was  first  indicator  of  target  appearance. 

♦♦No  crew  action  Indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions 
^System  acquired  target  prior  to  crew  action. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1027  (4  Fire  Unitsi  25 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

••No  crew  action  indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOB  I  Trial  1027  (4  Fire  Units}  25 
Engagements).  {All  times  in  seconds.) 
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Performance  Data  Base  (Cont'd. ) .  Target  engagement  time  durations  for  Sgt  York  FOB  I  Trial  1029  (3  Fire  Units}  12 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  Indicator  of  target  appearance. 

••No  crew  action  Indicated  In  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1035  (4  Fire  Units;  21 
Engagements).  (All  times  in  seconds.) 
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••No  crew  action  indicated  in  1553  data  base. 

{  ) Action  occurred  simultaneously  with  other  events/ functions, 
f System  acquired  target  independent  of  crew  action. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOB  I  Trial  1036  (4  Fire  Unitsi  13 
Engagements).  (Ml  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

••No  crew  action  Indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1039  <4  Fire  Units;  18 
Engagements).  (All  times  in  seconds.) 
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) Action  occurred  simultaneously  with  other  events/functions. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1039  (4  Fire  Unitst  18 
Engagements).  (All  times  in  seconds.) 
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Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1041  (3  Fire  Units*  16 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

•No  crew  action  indicated  in  1553  data  base. 

) Action  occurred  simultaneously  with  other  events/functions 
+No  trigger  release  between  targets. 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1042  (4  Fire  Units)  14 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  Indicator  of  target  a] 
•No  crew  action  indicated  in  1553  data  base. 


Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1046  (4  Fire  Onitsj  29 
.  (All  times  in  seconds.) 
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♦Crew  action  was  first  Indicator  of  target  appearance. 

**No  crew  action  indicated  in  1553  data  base. 

(  )  Act  ion  occurred  simultaneously  with  other  events/functions 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1046  (4  Fire  Units;  29 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  indicator  of  target  appearance. 

••No  crew  action  indicated  in  1553  data  base. 

(  ) Action  occurred  simultaneously  with  other  events/functions 


Performance  Data  Base  (Cont'd.).  Target  engagement  time  durations  for  Sgt  York  FOE  I  Trial  1048  (3  Fire  Units)  11 
Engagements).  (All  times  in  seconds.) 
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•Crew  action  was  first  Indicator  of  target  a] 
+No  trigger  release  between  targets. 


Table  4 


Mean  Durations  of  the  Component  Intervals  of  the  Target  Engagement  Sequence 
(plus  standard  deviation  and  N  ■  number  of  engagements) 


TES  Interval 

Mean 

Duration 

(secs) 

Standard 

Deviation 

(secs) 

N 

Select  Target 

7.9 

10.4 

221 

with  Radar  Pointing 

7.7 

9.4 

155 

with  Lasing 

8.3 

12.4 

66 

Pointing  or  Lasing  Target 

1.9 

1.8 

221 

Duration  of  Pointing 

2.3 

2.2 

155 

Duration  of  Lasing 

1.1 

0.9 

66 

Target  Acquisition 

4.2 

4.7 

271 

Via  Radar  Pointing 

4.9 

3.3 

155 

Via  Lasing 

0.1 

0.07 

66 

Via  Radar  Auto 

5.2 

2.3 

3 

System  Preempts 

6.6 

6.4 

43 

System,  then  Crew 

17.9 

10.6 

4 

Fire  Solution:  Blinking 

4.0 

3.1 

65 

Fire  Solution:  Solid 

2.6 

3.2 

206 

Fire  Solution:  B  +  S 

2.9 

3.2 

271 

Fire  Decision:  Blinking 

1.7 

2.5 

65 

Fire  Decision:  Solid 

1.3 

2.8 

206 

Fire  Decision:  B  +  S 

1.4 

2.7 

271 

Time  to  Fire 

16.5 

11.9 

271 

with  Radar  Pointing 

19.8 

11.1 

155 

with  Lasing 

14.3 

13.0 

66 

with  Radar  Auto 

6.7 

2.4 

3 

with  System  Preempts 

7.9 

6.9 

43 

with  System  Tgt  Acqu 

25.6 

11.8 

4 

Fire  Duration 

8.0 

8.0 

271 

Total  Engagement  Duration 

24.6 

14.3 

271 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action 


TRIAL  1020 


Fire  Unit  1 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 


Fire  Unit  3 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 


Fire  Unit  4 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 
Engmt.  8 
Engmt.  9 
Engmt.  10 
Engmt.  11 
Engmt.  12 
Engmt.  13 
Engmt.  14 
Engmt.  15 
Engmt.  16 
Engmt.  17 
Engmt.  18 
Engmt.  19 


Fire  Unit  5 


Engmt.  1 


TARGETS 

Scenario;  Attack 
See.  Tactic:  Havoc /Hover 


Targets  Target 
Displayed  Range 
0cm) 


SYSTEM  MODE 


67.6 

42.1 

20.1 
9.8 

36.0 

8.2 

9.4 


28.1 

13.4 

15.8 

36.8 

12.5 


17.0 

31.3 

14.6 

38.5 

25.6 

25.1 
5.9 

29.1 
8.2 

26.7 

6.3 
0.0 
2.0 

14.8 
18.6 
18.6 

11.8 

8.4 
74.6 


#  of 
Acq. 
Mode 


CREW 

ACTION- 


#  of  Mode  at  1st 

Non-Acq.  Trigger  Action 

Mode  Pull 


Table  S 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range ,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1021 


Fire  Unit  2 
Engmt.  1 


Fire  Unit  3 

Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 


Fire  Unit  5 

Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 


Tabl*  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1022  . 


Fire  Unit  2 


Engmt.  1 
Engmt.  2 


Fire  Unit  3 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 
Engmt.  8 
Engmt.  9 
Engmt.  10 
Engmt.  11 
Engmt.  12 
Engmt.  13 
Engmt.  14 
Engmt.  IS 
Engmt.  16 
Engmt.  17 
Engmt.  18 


Fire  Unit  4 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 


Fire  Unit  5 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 
Engmt.  8 
Engmt.  9 
Engmt.  10 


TARGETS 

Scenario:  Attack 

See.  Tactic:  Hind /Hover 


SYSTEM  MODE 


ACTION 


Targets  Target  #  of  #  of  Mode  at  1st 
Displayed  Range  Aeq.  Non-Acq.  Trigger  Action 
(lan)  Mode  Mode  Pull 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont’d.) 


TRIAL  1023 

TARGETS 

Scenario:  Delay 

See.  Tactic:  Havoc /Hover 

SYSTEM  MODE 

CREW 

ACTION 

El 

181 

Targets 

Displayed 

- 

Target 

Range 

Ocn) 

I 

ESI 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  2 

Engmt.  1 

19.4 

2 

0.5 

i 

1 

A 

G 

Engmt.  2 

19.7 

4 

1.0 

i 

1 

A 

G 

Engmt.  3 

6.3 

9 

0.0 

i 

1 

B 

G 

Engmt.  4 

5.4 

4 

3.0 

i 

1 

A 

SL 

Engmt.  S 

54.1 

9 

4.0 

10 

4 

9 

SL 

Engmt.  6 

4.7 

1 

o 

• 

o 

2 

1 

B 

G 

Fire  Unit  3 

■ 

■ 

Engmt.  1 

§BH 

3 

4 

2 

9 

SL 

Engmt.  2 

3BEI 

3 

1 

0 

B 

G 

Efcgmt.  3 

10.4 

8 

Isfl 

2 

1 

B 

G 

Fire  Unit  4 

Engmt.  1 

28.0 

6 

6.5 

2 

1 

A 

SL 

Engmt.  2 

35.3 

10 

8.0 

4 

3 

A 

SL 

Engmt.  3 

12.0 

4 

6.0 

1 

0 

A 

NA 

Engmt.  4 

14.9 

3 

5.0 

2 

2 

A 

SL 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1026 

TARGETS 

Scenario :  Road  March 

See.  Tactic:  Havoc /Hover 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

(s) 

Targets 

Displayed 

Target 

Range 

(km) 

#  of 
Acq. 
Mode 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

■ 

Engmt.  1 

13.6 

1 

■9 

2 

1 

A 

SL 

Engmt.  2 

0.0 

4 

■Sfl 

2 

0 

A 

NA 

Engmt.  3 

13.6 

3 

■a 

2 

2 

B 

G 

Fire  Unit  3 

Engmt.  1 

18.2 

8 

2.5 

1 

3 

B 

G 

Fire  Unit  4 

■ 

Engmt.  1 

5.3 

5 

D 

1 

1 

B 

G 

Fire  Unit  5 

Engmt.  1 

4 

0.0 

1 

0 

B 

G 

Engmt.  2 

6 

10.0 

2 

5 

B 

G 

Engmt.  3 

3 

7.0 

3 

0 

9 

NA 

Engmt.  4 

3 

3.0 

2 

B 

NA 

Engmt.  5 

24.6 

3 

5.0 

2 

9 

G 

Engmt.  6 

8.4 

8 

4.0 

3 

B 

G 

Engmt.  7 

40.6 

4 

5.0 

2 

0 

9 

NA 

Table  S 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1027 


Fire  Unit  1 


Engmt.  1 


Fire  Unit  3 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 
Engmt.  8 


Fire  Unit  4 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
Engmt.  7 
Engmt.  8 
Engmt.  9 
Engmt.  10 


Fire  Unit  5 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1029 

TARGETS 

Scenario:  Delay 

See.  Tactic:  Havoc/Hover 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

<s> 

Targets 

Displayed 

Target 

Range 

(km) 

#  Of 

Acq. 

Mode 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  2 

Engrat.  1 

10.3 

9 

3.5 

3 

1 

A 

SL 

Engmt.  2 

33.2 

5 

3.5 

2 

1 

9 

SL 

Engmt.  3 

11.3 

4 

5.0 

1 

0 

A 

SL 

Engmt.  4 

7.6 

10 

4.5 

1 

0 

A 

NA 

Engmt.  5 

17.5 

2 

wm 

3 

1 

A 

G 

Fire  Unit  4 

Engmt.  1 

14.6 

10 

2.0 

1 

1 

A 

SL 

Engmt.  2 

7.5 

6 

2.0 

1 

1 

A 

SL 

Engrat.  3 

36.1 

1 

8.5 

1 

1 

A 

SL 

Fire  Unit  5 

Engmt.  1 

6 

4.0 

1 

0 

8 

NA 

Engmt.  2 

15.4 

8 

3.5 

4 

1 

A 

SL 

Engmt.  3 

10.8 

8 

5.0 

2 

1 

A 

NA 

Engmt.  4 

17.7 

4 

6.0 

5 

1 

9 

SL 

Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range /  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1035 

TARGETS 

Scenario:  Delay 

See.  Tactic:  Havoc/Hover 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

-  is)- 

Targets 

Displayed 

Target 

Range 

(km) 

1  #  of 
Acq. 
Mode 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

Engmt.  1 

5.9 

1 

0.0 

1 

0 

B 

G 

Engmt.  2 

23.0 

5 

5.0 

2 

1 

9 

NA 

Engmt.  3 

12.2 

4 

5.5 

1 

1 

B 

G 

Engmt.  4 

13.1 

4 

5.5 

3 

1 

A 

SL 

Engmt.  5 

4.3 

11 

3.0 

1 

1 

B 

G 

Engmt.  6 

9.2 

7 

3.5 

2 

1 

A 

SL 

Engmt.  7 

13.6 

11 

5.0 

2 

1 

A 

SL 

Engmt.  8 

51.0 

8 

4.0 

1 

3 

B 

G 

Fire  Unit  2 
■ . . 

Engmt.  1 

4.5 

1 

0.0 

0 

B 

G 

Engmt.  2 

22.9 

6 

3.5 

4 

B 

G 

Engmt.  3 

16.7 

4 

5.0 

2 

C 

NA 

Engmt.  4 

25.3 

7 

5.0 

1 

A 

G 

Engmt.  5 

26.8 

3 

6.0 

2 

1 

9 

NA 

Fire  Unit  3 

Engmt.  1 

11.2 

4 

6.0 

■ 

1 

A 

G 

Engmt.  2 

9.9 

2 

2.0 

1 

A 

G 

Engmt.  3 

12.4 

3 

2.0 

1 

A 

SL 

Engmt.  4 

48.8 

2 

2.0 

2 

9 

SL 

Engmt.  5 

30.3 

3 

4.5 

n 

4 

A 

G 

Fire  Unit  4 

Engmt.  1 

4 

5.0 

i 

1 

A 

SL 

Engmt.  2 

■eeI 

12 

3.0 

2 

0 

A 

NA 

Engmt.  3 

14.9 

2 

4.0 

1 

1 

A 

G 

Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


1 

1 

CREW 

ACTION 

TRIAL  1036 

TARGETS 

Scenario:  Attack 

See.  Tactic:  Havoc /Hover 

SYSTEM  MODE 

Time 

to 

Fire 

(s) 

Targets 

Displayed 

Target 

Range 

(km) 

#  of 
Acq. 
Mode 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

Engmt.  1 

35.7 

2 

9.0 

2 

2 

A 

G 

Engmt.  2 

11.1 

4 

3.0 

2 

2 

A 

NA 

Engmt.  3 

31.1 

4 

3.0 

3 

2 

B 

G 

Engmt.  4 

18.3 

2 

0.0 

3 

1 

9 

G 

Fire  Unit  2 

Engmt.  1 

59.1 

1 

1.0 

1 

4 

B 

G 

Fire  Unit  3 

Engmt.  1 

11.8 

2 

4.5 

1 

1 

A 

G 

Engmt.  2 

51.0 

1 

4.0 

2 

1 

9 

G 

Fire  Unit  4 

Engmt.  1 

11.2 

2 

7.0 

1 

1 

A 

G 

Engmt.  2 

11.5 

7 

2.0 

1 

1 

A 

G 

Engmt.  3 

11.4 

4 

5.5 

1 

1 

A 

G 

Engmt.  4 

26.0 

7 

4.5 

2 

2 

A 

SL 

Er^mt.  5 

15.5 

7 

2 

1 

A 

SL 

Engmt.  6 

9.0 

6 

Kai 

Hi 

1 

A 

SL 

Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont’d.) 


TRIAL  1039 

TARGETS 

Scenario:  Attack 

See.  Tactic:  Havoc /Hover 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

(s) 

Targets 

Displayed 

Target 

Range 

(km) 

#  of 
Acq. 
Mode 

#  of 
Non- Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

Engmt.  1 

11.5 

4 

7.0 

1 

1 

A 

G 

Engmt.  2 

20.7 

3 

5.0 

1 

1 

A 

SL 

Engmt.  3 

31.1 

1 

2.5 

3 

4 

B 

SL 

Engmt.  4 

26.3 

2 

3.5 

4 

5 

A 

G 

Fire  Unit  2 

Engmt.  1 

13.3 

4 

9.0 

2 

1 

9 

G 

Engmt.  2 

22.1 

7 

3.5 

3 

2 

A 

SL 

Engmt.  3 

34.7 

4 

8.5 

3 

4 

9 

SL 

Engmt.  4 

9.5 

8 

1.0 

2 

1 

A 

SL 

Fire  Unit  3 

Engmt.  1 

23.5 

5 

10.0 

3 

1 

9 

SL 

Engmt.  2 

43.9 

1 

7.5 

2 

1 

9 

SL 

Engmt.  3 

19.1 

3 

4.0 

5 

2 

B 

G 

Engmt.  4 

12.9 

3 

1.5 

3 

2 

B 

G 

Engmt.  5 

15.5 

5 

6.0 

3 

3 

A 

SL 

Fire  Unit  4 

Engmt.  1 

20.4 

1 

4.0 

1 

2 

A 

G 

Engmt.  2 

24.1 

3 

3.5 

2 

2 

A 

SL 

Engmt.  3 

24.8 

1 

4.0 

2 

2 

B 

G 

Engmt.  4 

6.6 

3 

3.5 

■M 

1 

B 

G 

Engmt.  5 

2.4 

2 

o 

• 

o 

HI 

0 

B 

i _ 

G 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1040  ;• 

TARGETS 

Scenario:  Road  March 

See.  Tactic:  Hind/Hover 

SYSTEM  M3DE 

CREW 

ACTION 

Time 
to 
Fire 
_ ill— 

Targets 

Displayed 

Target 

Range 

(km) 

#  of 
Acq. 
Mode 

#  of 
Non-Acq. 
Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  2 

Engmt.  1 

23.1 

4 

5.0 

1 

1 

A 

G 

Engmt.  2 

18.6 

4 

4.0 

1 

0 

A 

SL 

Engmt.  3 

15.9 

4 

4.0 

1 

1 

A 

G 

Fire  Unit  4 

Engmt.  1 

12.7 

1 

1 

1 

A 

G 

Engmt.  2 

32.2 

1 

2 

2 

A 

SL 

Engmt.  3 

19.4 

2 

3 

2 

A 

G 

Engmt.  4 

18.9 

1 

3.5 

2 

1 

A 

SL 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1041 

TARGETS 

Scenario:  Attack . 

See.  Tactic:  Hird /Running 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

(s; 

Targets 

Displayed 

Target 

Range 

(to) 

*  of 

A cq. 
Mode 

#  of 

Non-Acq. 

Mode 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

Engmt.  1 

11.4 

2 

4.0 

1 

1 

A 

SL 

Engmt.  2 

35.7 

5 

4.0 

1 

3 

B 

G 

Fire  Unit  3 

Engmt.  1 

35.2 

4 

6.5 

2 

2 

A 

SL 

Engmt.  2 

12.4 

2 

5.5 

1 

1 

A 

G 

Fire  Unit  5 

Engmt.  1 

9.2 

1 

0.0 

1 

1 

B 

G 

Engmt.  2 

9.6 

7 

1.0 

1 

1 

A 

SL 

Engmt.  3 

3.0 

6 

3.0 

1 

0 

9 

NA 

Eigmt.  4 

1.7 

7 

3.0 

1 

0 

9 

NA 

Engmt.  5 

8.7 

5 

5.5 

1 

0 

B 

G 

Engmt.  6 

17.1 

6 

6.5 

4 

0 

A 

G 

Engmt.  7 

32.7 

2 

3.5 

4 

3 

9 

SL 

Engmt.  8 

13.0 

2 

■ 

3 

1 

9 

SL 

Engmt.  9 

8.8 

6 

1 

G 

Engmt.  10 

6.4 

4 

1 

SL 

Engmt.  11 

8.6 

2 

■  .  1 

NA 

Engmt.  12 

2.9 

3 

Bsfl 

II 

HU 

NA 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TARGETS  SYSTEM  MODE 

Scenario :  Delay 

See.  Tactic:  Hind/Running 

TRIAL  1042  - - - - - 

Time  Targets  Target  #  of  #  of  Mode  at  1st 
to  Displayed  Range  Acq.  Non-Acq .  Trigger  Action 
Fire  (ton)  Mode  Mode  Pull 


Fire  Unit  1 

Engmt.  1 
Engmt.  2 


Fire  Unit  2 


Engmt.  1 
Engmt.  2 


Fire  Unit  4 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
Engmt.  5 
Engmt.  6 


Fire  Unit  5 


Engmt.  1 
Engmt.  2 
Engmt.  3 
Engmt.  4 
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Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1046 

TARGETS 

Scenario:  Attack 

See.  Tactic:  Hind/Hover 

SYSTEM  MODE 

CREW 

ACTION 

Time 

to 

Fire 

(s) 

Targets 

Displayed 

Target 

Range 

(km) 

#  of 
Acq. 
Made 

#  Of 
Non-  Acq. 
Made 

Mode  at 
Trigger 
Pull 

1st 

Action 

Fire  Unit  1 

■ 

Engmt.  1 

11.2 

2 

1.0 

H 

3 

B 

G 

Engmt.  2 

9.5 

3 

4.5 

2 

A 

SL 

Engmt.  3 

6.4 

11 

3.5 

1 

1 

A 

SL 

Engmt.  4 

23.4 

1 

3.0 

■ 

2 

A 

SL 

Engmt.  5 

23.4 

1 

5.5 

1 

A 

SL 

Engmt.  6 

0.0 

3 

i 

5.5 

H 

0 

A 

NA 

Fire  Unit  2 

Engmt.  1 

14.4 

1 

6.0 

1 

A 

G 

Engmt.  2 

22.1 

4 

2.0 

2 

A 

G 

Engmt.  3 

18.2 

7 

2.5 

1 

A 

G 

Engmt.  4 

18.2 

3 

4.0 

2 

A 

G 

Engmt.  5 

10.0 

6 

■ 

2 

A 

NA 

Engmt.  6 

5.7 

5 

2 

1 

B 

G 

Engmt.  7 

34.5 

17 

10.0 

3 

2 

B 

G 

Engmt.  8 

41.9 

6 

6.0 

2 

4 

B 

G 

Fire  Unit  3 

Engmt.  1 

9.2 

5 

1.5 

1 

1 

A 

SL 

Engmt.  2 

21.1 

6 

2.0 

1 

0 

A 

G 

Engmt.  3 

2.4 

3 

1.0 

1 

0 

A 

NA 

Fire  Unit  5 

Engmt.  1 

11.1 

5 

7.0 

1 

1 

A 

SL 

Engmt.  2 

6.0 

5 

0.0 

1 

1 

B 

G 

Engmt.  3 

9.3 

2 

2.0 

1 

1 

A 

SL 

Engmt.  4 

6.8 

4 

2.0 

1 

1 

A 

NA 

Engmt.  5 

8.4 

5 

2.0 

1 

2 

B 

G 

Engmt.  6 

20.9 

6 

3.0 

3 

4 

C 

G 

Engmt.  7 

10.4 

2 

1 

1 

A 

G 

Engmt.  8 

15.5 

3 

0 

A 

SL 

Engmt.  9 

12.2 

9 

0 

A 

SL 

Engmt.  10 

10.4 

5 

3 

B 

G 

Engmt.  11 

24.0 

6 

3 

A 

SL 

Engmt.  12 

10.2 

6 

HI 

2 

0 

B 

G 

Table  5 


Target  Engagement  Tasks  Showing  Targets  Displayed  and  Range,  System  Modes,  and 
Crew  Action  (Cont'd.) 


TRIAL  1048 

TARGETS 

Scenario:  Attack 

See.  Tactic:  Havoc/Hover 

SYSTEM  MODE 

CREK 

ACTION 

Time 

Targets 

Target 

#  of 

#  of 

Mode  at 

1st 

to 

Displayed 

Range 

Acq. 

Non-Acq. 

Trigger 

Action 

Fire 

(Jon) 

Mode 

Mode 

Pull 

(s) 

Fire  Unit  3 

■ 

Engmt.  1 

4.5 

5 

1 

0 

8 

NA 

Engmt.  2 

6.5 

4 

2 

0 

8 

NA 

mm 

Engmt.  1 

7.1 

8 

2.5 

2 

2 

c 

G 

Engmt.  2 

5.6 

1 

0.0 

1 

0 

B 

G 

Engmt.  3 

17.6 

1 

0.0 

3 

0 

B 

G 

Engmt.  4 

12.1 

5 

5.5 

1 

1 

A 

SL 

Fire  Unit  5 

Engmt.  1 

15.9 

7 

4.0 

2 

4 

A 

SL 

Engmt.  2 

8.8 

6 

1.0 

1 

3 

B 

G 

Engmt.  3 

10.6 

9 

3.5 

2 

1 

A 

SL 

Engmt.  4 

28.6 

8 

3.5 

2 

1 

B 

G 

Engmt.  5 

9.7 

2 

7.0 

1 

1 

A 

SL 
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Missing  Data  Analyses 


The  approach  taken  in  compiling  the  data  base  presented  in  the  preceding 
section  represents  one  of  many  possible  ways  of  looking  at  the  data  available 
on  system  (man  and  machine)  performance  during  the  Sgt  York  FOE  I  test.  The 
choice  was  made  to  analyze  only  target  engagements  that  included  firing,  the 
terminal  task  of  the  target  engagement  sequence. 

As  the  data  on  completed  engagement  sequences  were  extracted,  it  became 
apparent  that  many  engagement  sequences  had  been  interrupted  before  firing 
occurred.  It  would  have  been  possible  to  collect  data  on  such  Interrupted 
sequences  for  each  trial,  and  each  fire  unit,  and  to  categorize  each  one  by 
the  stage  at  which  the  interruption  or  break-off  occurred.  That  is,  it  would 
have  been  possible  to  do  a  great  deal  of  what  might  be  considered  error 
analysis.  As  noted  earlier,  a  blinking  fire  cue  did  not  constitute  a 
mandatory  firing  situation,  but  a  solid  fire  cue  was  a  clear  indication  that  a 
target  should  be  fired  on.  A  crew  member  might  have  chosen  not  to  fire  if, 
for  example,  his  visual  check  of  the  target  dlsconfirmed  the  IFF  classifica¬ 
tion  of  the  target  as  a  foe.  In  such  a  case,  the  operator  could  have  used  the 
break-off  switch  and  indicated  that  the  target  was  a  friend. 

Without  such  a  legitimizing  explanation,  not  firing  on  a  solid  fire  cue 
could  be  considered  an  error.  The  information  provided  by  such  analyses  would 
constitute  an  alternative  and  additional  view  of  operator  and  crew  perfor¬ 
mance.  For  example,  an  exploratory  error  analysis  of  a  single  trial  (Trial 
1036)  revealed  that  Fire  Unit  1  fired  on  all  solid  firing  cues  throughout  the 
trial.  Fire  Units  2  and  3  each  had  one  solid  fire  cue  and  in  each  case  the 
respective  crews  failed  to  fire  on  the  target.  Fire  Unit  4,  the  only  other 
fire  unit  involved  in  Trial  1036,  had  solid  fire  cues  presented  in  two 
separate  instances  and  in  neither  case  did  the  crew  fire.  The  crew  did  point 
and  engage  the  target  but  did  not  take  the  final  step  of  pressing  the  trigger. 

One  possible  explanation  for  a  failure  to  activate  the  trigger  and  fire  on  a 
target  is  that  the  time  available  was  too  brief.  As  explained  above,  error 
analysis  was  not  the  focus  of  this  study,  and  once  it  was  determined  that  an 
emphasis  on  complete  target  engagement  sequences  would  provide  more  useful 
data  in  terms  of  applicability  to  future  FAADS  modeling  efforts,  the  con¬ 
sideration  of  incomplete  engagements  was  put  aside.  Preliminary  as  they  are, 
the  data  from  three  trials  showing  by  trial  and  by  fire  unit  how  long  solid 
fire  cues  were  displayed  to  the  crews  as  part  of  engagements  that  were  never 
carried  through  to  trigger  pull  are  presented  in  Table  6.  An  early,  a  mid¬ 
term,  and  a  late  trial  were  chosen  to  see  if  there  was  evidence  of  change  in 
number  of  fire  cues  to  which  crews  did  not  respond  as  FOE  I  progressed. 

There  does  seem  to  be  some  evidence  of  improvement.  Whether  or  not  Fire  Unit 
4  in  Trial  1036  had  Time  to  Fire  on  the  target  in  the  two  instances  noted 
above,  the  fact  remains  that  they  did  not.  Despite  this  non-firing.  Fire  Unit 
4  in  each  of  the  two  instances  received  feedback  that  the  target  was  presumed 
to  have  been  damaged. 

Being  able  to  damage  a  target  without  firing  on  it  raises  some  questions  of 
validity  that  deserve  attention.  In  contrast  to  battlefield  conditions,  the 
Force-on-Force  trials  provided  no  actual  target  destruction  to  provide 
knowledge  of  results  and  to  signal  that  firing  could/should  cease.  The 
theoretical  success  of  an  engagement  for  these  Force-on-Force  trials  was 
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Table  6 


Total  Number  of  Seconds  Solid  Firing  Cues  Were  Displayed  for  Targets  on 
Milch  No  Firing  Occurred 


TRIAL  1020 


Fire  Unit  1 

Fire  Unit  2  Fire  Unit  3 
>  . .  '  11 

Fire  Unit  4 

Fire  Unit  5 

1.8 

N/A  1.1 

35.3 

N/A 

10.9 

2.7 

12.1 

4.3 

2.7 

7.1 

1.1 

0.0 

1.4 

1.1 

3.2 


TRIAL  1036 

Fire  Unit  1  Fire  Unit  2  Fire  Unit  3  Fire  Unit  4  Fire  Unit  5 

0.0  10.1  3.3  1.0  N/A 

1.7 


TRIAL  1048 

Fire  Unit  1  Fire  Obit  2  Fire  Unit  3  Fire  Obit  4  Fire  Obit  5 


determined  by  a  Monte  Carlo  model  available  to  CDEC.  Evidently  in  the  case 
of  the  Sgt  York  FOE  I,  either  trigger  pull  was  not  an  element  required  for 
successful  engagement  according  to  the  model  used,  or  in  instances  such  as  the 
ones  cited  above  for  Fire  Unit  4  in  Trial  1036,  some  error  occurred.  That  it 
was  not  an  isolated  or  uncommon  occurrence  was  evident  from  an  examination  of 
the  1553  printouts. 

The  results  of  the  Monte  Carlo  damage  assessment  were  supposed  to  be  relayed 
to  crew  members,  but  the  information  transmission  was  often  delayed  and 
sometimes  never  occured.  In  addition,  problems  of  credibility  arose  when  a 
crew  was  told  it  had  damaged  a  target  it  had  not  fired  on  (and  conversely  that 
it  had  failed  to  damage  a  target  that  the  crew  believed  "killed"). 

More  often  than  not,  no  record  of  the  outcome  was  included  as  part  of  the 
printout  of  the  1553  data  bus;  thus,  no  outcome  information  was  available  for 
this  data  analysis.  Had  such  information  been  available,  something  in 
addition  to  a  simple  time-to-perform  measure  could  have  been  developed, 
something  that  would  make  it  possible  to  judge  whether  or  not  the  fastest 
engagements  were  the  most  effective.  Any  engagement  to  be  effective  must 
reach  Time  to  Fire  before  the  hostile  target  destroys  the  defense,  but  if  the 
firing  does  not  actually  damage  the  target,  the  engagement  should  not  be 
judged  a  success  no  matter  how  quickly  the  trigger  was  pulled. 

The  preceding  discussions  suggest  the  richness  of  the  unexplored  data.  The 
next  chapter  will  illustrate  the  variety  of  what  was  explored.  In  that 
chapter,  fourteen  topics  are  discussed.  The  discussions  are  based  on  analyses 
of  the  data  contained  in  Tables  3  and  5.  They  deal  with  tactical  and  system 
parameters  and  their  relationship  to  individual,  crew,  and  system  performance. 
The  analyses  presented  do  not  begin  to  exhaust  the  possibilities  that  exist. 
Many  additional  comparisons  could  be  made.  It  is  our  intent  in  providing 
these  data  to  give  interested  designers  and  researchers  real  numbers  based  on 
the  actual  performance  of  crews  using  an  operating  system  in  a  field  test 
environment. 
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Research  Findings 


The  data  presented  in  Tables  3  and  5  are  an  Indication  of  the  quantity  of 
raw  performance  data  available  for  analysis  and  Interpretation.  In  order  to 
bring  some  structure  to  these  data  and  the  areas  they  represent,  five 
categories  were  established  as  follows: 

•  System  and  Subsystem  performance, 

•  Tactical  performance  variations, 

•  Individual  and  crew  performance, 

•  Personnel  quality,  and 

•  Training. 

In  the  discussions  that  follow  in  this  chapter,  each  of  these  major  areas 
will  be  addressed  with  appropriate  supporting  data. 

System  and  Subsystem  Performance.  Three  topics  are  covered  under  this 
general  area: 

•  System  and  subsystem  response  times 

•  System  modes  and  levels  of  automation 

•  Fire  unit  comparisons 

Two  themes  of  major  importance  are  (1)  what  levels  of  systems  and  subsystem 
performance  were  achieved  and  (2)  the  impact  of  manual  and  semi-automated 
system  modes  of  operation. 

Tactical  performance  variations.  Forward  area  air  defense  offers  a  very 
rich  environment  for  performance  variations  as  a  function  of  different 
scenario  and  battle  conditions.  Data  are  provided  here  for  a  number  of 
variables : 

•  Varying  scenario  types 

•  Rotary-wing  tactics 

•  ECM  conditions 

•  Time  of  day,  and 

•  Target  range  at  first  appearance. 

These  data  provide  a  rich  source  of  information  for  the  interaction  of 
tactics,  crew  performance,  and  subsystem  performance. 

Individual  and  crew  performance.  An  unusual  quanitity  of  data  is 
available  for  the  performance  of  individuals  and  two-man  crews.  Three  topics 
are  discussed  here: 

•  Crew  action  and  decision  response  times 

•  Mission-oriented  protective  posture  (MOPP) 

•  Target  workload 

Detailed  information  is  available  about  many  of  the  actions  the  individuals 
and  crews  took,  as  well  as  how  they  used  the  system. 
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Personnel  quality.  In  this  specific  test  case,  much  was  known  about  the 
personnel  who  participated  in  the  test.  This  information  is  related  to 
performance  and  presented  in  two  parts: 

•  Personnel  characteristics  and  individual  performance 

•  Crew  mix,  mental  categories,  and  crew  performance 

It  may  be  expected  that  in  future  FAADS  design  much  more  attention  will  be 
given  to  the  relationship  between  predicted  personnel  quality  and  appropriate 
design  features,  due  to  the  MANPRINT  initiative  (Army  Regulation  602-2,  1987). 

Training .  Although  the  data  are  limited,  some  comparisons  can  be  made 
with  respect  to  such  factors  as  previous  system  experience  or  the  lack  of  it, 
individual  training  scores,  collective  training  outcomes,  and  mean  performance 
times  for  such  operator  activities  as  selecting  and  classifying  targets, 
pointing  targets,  and  lasing  targets. 

Data  files  and  analysis.  It  would  not  have  been  feasible  to  process  these 
data  without  computer  assistance.  Appendix  A  provides  a  description  of  the 
data  files  and  the  data  processing  method.  The  data  files  were  structured 
redundantly,  so  they  can  be  processed  by  either  dBase  II  or  III.  The  data 
base  is  available  on  floppy  disc  for  MS-DOS  and  IBM-cinoatubke  personal 
computers.  The  statistical  analysis  software  package  was  SYSTAT.  The 
correlations  among  parameter  measures  may  be  of  some  psychometric  technical 
interest.  The  correlation  matrix  is  shown  and  discussed  In  Appendix  B. 


System  and  Subsystem  Performance 


System/Subsystem  Response  Times 


In  1976  the  Army  established  a  Required  Operational  Capability  (ROC)  for 
a  new  air  defense  gun  (USA-TRADOC  ACN22087).  The  system  would  be  required  to 
be  able  to  fire  within  7  seconds  at  any  target  following  its  detection  and 
establishment  as  a  hostile  target. 

So,  one  of  the  most  important  aspects  of  the  Sgt  York's  performance 
during  the  Force-on-Force  trials  is  the  length  of  time  it  took  to  start  firing 
at  hostile  targets;  i.e.,  how  long  was  the  system  Time  to  Fire?  Does  the 
observed  Time  to  Fire  meet  the  7  second  requirement? 

Figure  21  shows  the  overall  means  of  segments  of  system  performance 
derived  from  the  engagements  selected  for  purposes  of  this  study/analysis . 

From  these  data,  it  can  be  noted  that  the  overall  mean  Time  to  Fire,  based  on 
271  target  engagement  sequences,  was  16*5  seconds.  This  does  not  meet  the 
operational  requirement  of  7  seconds;  it  is  more  than  twice  that  amount  of 
time. 
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6.4  secs,  if  N  -  271 


1.  Target  Appearance 

1  to  2:  7.9  secs.  (N  -  221) 

2.  Crew  pushes 
Radar  or  Laser  Button 

2  to  3:  1.9  secs.  (N  ■  221) 

3.  Computer  releases 
Radar  or  Laser  Button 

3  to  4:  3.5  secs.  (N  -  221) 

1  to  2  to  3  to  4:  13.3  secs.  (N  *  221) 
1  to  4:  7.4  secs.  (N  •  50) 

4.  Radar  or  Laser  Cue  Appears 

4  to  5:  4.0  secs.  (N  *  271) 

5.  Fire  Solution  Cue(s)  Appear(s) 

5  to  6:  1.4  secs.  (N  -  271) 

6.  Crew  pulls  Trigger 

Time  To  Fire  (no  steps  2  &  3)  13.1  secs.  (N  ■  50  &  271) 

Time  To  Fire  (N  -  271  for  each  step)  16.5  secs. 

Time  To  Fire  (all  steps  included)  19.0  secs.  (N  *  221  or  271) 


Note.  The  Required  Operational  Capability  is  a  Time  To  Fire  of  7  seconds 


Figure  21.  System  and  subsystem  performance  times  for  indicated  numbers  of 
engagements. 
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It  is  also  worthwhile  to  look  within  the  target  engagement  sequence  at 
component  subsystem  response  times  as  shown  in  Figure  21: 

1.  It  took  a  mean  of  6.4  seconds  (computed  for  271  engagements)  to 
acquire  and  Identify  the  target.  This  step  was  a  combined  manual-computer 
function  which  included  the  essential  IFF  function  as  a  principally  automated 
mode  and  a  variety  of  manual  and  semi-automatic  control  modes.  This  component 
mean  alone  consumes  almost  all  of  the  prescribed  system  Time  to  Fire.  Perhaps 
worse,  the  above  mean  included  50  enagagements  for  which  there  was  no  event 

#2  because  of  system  program  intervention.  For  the  221  engagements  wherein 
the  crew's  action  ended  the  Target  Acquisition  interval,  the  mean  time 
required  was  7.9  seconds. 

2.  In  221  engagements,  the  crew  member  either  used  Radar  pointing  or 
lased  the  target  for  an  input  to  enable  the  display-computer  system  to  compute 
a  fire  solution.  This  step  measures  the  length  of  time  it  took  the  operator 
to  get  the  computer  to  accept  his  designation-location  of  the  target.  The 
mean  pointing/lasing  time  was  1.9  seconds  for  N  «  221.  (Computed  for  N  *  271, 
the  mean  was  1.6  seconds.) 

3.  Automated  decision  support  was  provided  by  a  fire  control  computer 
subsystem,  coupled  with  an  automatic  tracking  system.  Figure  21  shows  that 
the  mean  time  consumed  by  automated  decision  support  at  this  point  in  the 
target  engagement  sequence  was  7.4  seconds  (Steps  1  to  4,  n  *  50).  This  also 
is  more  than  the  required  system  Time  to  Fire  of  7  seconds. 

4.  There  had  to  be  a  manual  (crew)  decision  to  fire.  For  n  -  271,  the 
mean  time  to  make  the  decision  to  fire  was  1.4  seconds. 

In  short,  the  system  was  not  able  to  achieve  on  the  average,  over  a  large 
sample  of  engagements,  the  required  system  performance  response  time. 

Some  of  the  variations  or  modes  of  system  operation  took  less  time  than 
others,  and  these  variations  are  presented  in  later  paragraphs. 

In  the  evolving  design  of  semi-automated  systems,  there  have  been  several 
occasions  where  automating  has  imposed  greater  burdens  on  the  system  and 
impeded  both  load  processing  (such  as  number  of  targets  processed)  and 
timeliness  (such  as  Time  to  Fire).  This  probably  was  inevitable  as  technology 
was  developed  in  a  piece-meal  fashion  to  transform  subsystem  functions  from 
manual  to  semi-automatic  and  automatic.  It  is  important  in  each  case  to 
estimate  closely  how  well  a  subsystem  will  perform  as  a  part  of  the  total 
system.  "Optimal"  subsystems,  even  if  achieved,  do  not  necessarily  induce 
optimum  system  performance. 

The  results  shown  in  Figure  21  are  Illustrative  of  many  past  and  present 
systems  where  manual  and  automated  functions  have  either  been  joined  or 
integrated.  The  results  are  indicative  of  at  least  four  continuing  technology 
problems : 

1.  However  good  a  subsystem  function  "solution”  may  be  technically, 
total  system  performance  effectiveness  may  not  necessarily  be  enhanced.  Thus, 
during  design,  there  is  an  urgent  need  for  a  system  engineering  approach  to 
make  sure  that  the  parts  add  up  to  an  acceptable,  effective  whole.  In  the 
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very  short  response  times  required  by  air  defense,  careful  attention  must  be 
given  to  each  subsystem  to  insure  acceptable  system  response  times. 

2.  It  is  important  that  sufficient  simulation  and  developmental  testing 
be  performed  before  prototype  operational  testing  occurs.  In  developmental 
testing,  assurance  can  be  reached,  or  correction  can  be  made,  so  that  required 
system  and  subsystem  response  times  can  be  achieved  before  design  freeze  and 
actual  hardware  development.  It  is  simply  too  late  to  find  out  in  operational 
test  that  the  system  did  not  meet  operational  requirements. 

3.  Allocation  of  function  to  man  and  machine  is  still  an  uncertain  art 
(Price,  1985;  Kantowitz  &  Sorkin,  1987).  The  allocations  in  this  system 
seemed  appropriate,  yet  the  combined  human-automatic  target  analysis  step 
consumed  virtually  the  entire  response  time  requirement.  The  automated 

decision  support  subsystem  processing  time  exceeded  the  required  Time  to 
Fire.  However  appealing  automated  decision  aids  may  be,  they  must  meet  system 
performance  requirements . 

4.  The  required  Time  to  Fire  for  this  battlefield  system  is  short,  yet 
reflects  the  demands  of  the  modern  battlefield.  But,  as  a  technical  and 
operational  standard,  the  simple  number  of  7  seconds  leaves  something  to  be 
desired.  It  is  probable  that  a  set  of  performance  requirements  varying  as  a 
function  of  battlefield  scenarios  and  conditions  would  be  more  useful  for 
design  and  should  be  available  from  the  operational  analysis  of  future 
battlefield  scenarios. 

It  may  be  of  value  to  consider  the  potential  consequences  of  a  response 
time  such  as  that  shown  in  Figure  21.  If  the  system  can  only  respond  on  the 
average  within  16-17  seconds,  one  question  could  be:  What  would  be  the 
predicted  consequences  of  such  a  response  time?  It  may  be  that  there  are  some 
battlefield  conditions  where  such  response  times  might  be  acceptable.  A  study 
of  this  could  be  done  by  using  the  data  of  this  report  in  a  computer  model  of 
future  FAAD  systems. 

System  Modes  and  Levels  of  Automation 

To  evaluate  the  effect  of  the  use  of  the  different  system  modes  available 
to  Sgt  York,  several  different  comparisons  were  made.  For  the  first  com¬ 
parison,  three  nominal  system  modes  were  used:  Radar  Auto,  Radar  Pointer, 
and  (Radar)  Optical.  For  the  first  comparison,  laser  capability  was  dis¬ 
regarded;  that  is,  all  engagements  using  radar  pointer,  whether  with  or 
without  laser,  were  combined,  as  were  all  nominally  optical  engagements.  For 
the  second  comparison,  the  modes  were  broken  down,  as  far  as  possible,  into 
five  individual  modes;  that  is,  radar  pointer  engagements  with  laser  capabil¬ 
ity  were  separated  from  radar  pointer  engagements  without  laser  capability. 

For  the  third  comparison,  all  laser  modes  were  combined  and  contrasted  with 
non-laser  modes.  For  the  fourth  and  final  comparison,  a  new  structure  was 
created  based  on  the  way  system  events  occured  during  some  of  the  Force-on- 
Force  trials;  i.e.,  in  ways  not  described  in  pre-FOE  documents. 

For  each  of  the  comparisons,  the  target  engagement  sequences  were  broken 
down  by  acquisition  mode  into  the  mean  durations  of  the  eleven  component 
intervals  discussed  earlier:  Select,  Pointer  On,  Laser  On,  Acquisition  Time, 
Fire  Solution  Blink,  Fire  Solution  Solid,  Fire  Duration  Blink,  Fire  Duration 
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Solid,  Time  to  Fire,  Fire  Duration,  and  Engagement  Time  (not  a  component,  of 
course).  Throughout  this  report,  in  evaluating  differences  in  mean  times,  a 
probability  level  of  .05  has  been  adopted  as  the  criterion  for  significance  of 
the  differences. 

To  implement  making  the  comparisons  discussed  above,  the  data  were  con¬ 
figured  into  the  following  combinations: 

1.  Modes  Comparison  -  Nominal  System  Modes 

a.  Mode  8  -  Radar  Auto 

b.  Modes  9  &  A  -  Radar  Pointer 

c.  Modes  B  &  C  -  (Radar)  Optical 

2.  Modes  Comparison  -  Individual  Modes 

a.  Mode  8  -  Radar  Auto 

b.  Mode  9  -  Laser  Track,  Radar  Pointer 

c.  Mode  A  -  Radar  Pointer 

d.  Mode  B  -  Laser  Track,  Radar,  Optical 

e.  Mode  C  -  Radar,  Optical 

3.  Modes  Comparison  -  Effect  of  Laser 

a.  With  Laser 

b.  Without  Laser 

A.  Levels  of  Automation  Comparisons 

a.  Radar  Auto 

b.  System  Preempts 

c.  Radar  Pointer  Primary 

d.  Lase  Primary 

In  comparison  1,  of  nominal  system  modes,  the  category  of  Radar  Auto 
included  only  Mode  8.  The  second  category  combined  Mode  9,  Radar  Pointer  with 
Laser  Track,  with  Mode  A,  Radar  Pointer  without  Laser.  Category  three 
included  both  optical  modes  (Modes  B  &  C) .  This  comparison  of  nominal  system 
modes  found  two  components  of  the  engagement  sequence  to  be  significantly 
different.  Target  Acquisition  time  (p<.001)  and  Fire  Duration  time  (p*.028). 

As  Table  7  shows,  for  Target  Acquisition  time,  the  Optical  Mode,  with  a 
mean  of  1.5,  had  a  significantly  shorter  mean  than  the  Radar  Pointer  Modes' 
mean  of  5. A  seconds  and  the  Radar  Auto  Mode  mean  of  5.2  seconds.  In  this 
context,  it  is  important  to  note  the  difference  in  what  acquisition  time 
represents  in  the  various  modes.  Ordinarily,  acquisition  time  represented  the 
interval  between  the  time  when  the  radar  or  laser  button  was  released  (i.e., 
the  target  had  been  pointed  or  lased)  and  the  time  when  the  radar  or  laser  cue 
came  on.  However,  in  the  Auto  mode,  there  was  no  crew  pointing  or  lasing;  for 
that  mode,  acquisition  was  measured  as  the  interval  from  target  appearance 
time  to  the  onset  of  the  radar  cue.  (This  is  also  the  case  for  the  category 
of  System  Preempts,  discussed  later.)  For  the  Radar  Pointer  and  Lase 
categories,  Target  Acquisition  is  the  interval  from  the  computer's  release  of 
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the  Radar  or  Laser  buttons  until  the  Radar  or  Laser  cue  appears  on  the 
display. 


Table  7 

Modes  Comparison,  Nominal  System  Modes,  Target  Acquisition 


ENGAGEMENTS 

(n) 

MEAN 

(8) 

S.D. 

(s) 

Radar  Auto:  Mode  8 

3 

5.2 

2.3 

Radar  Pointer 

(with  or  without 

laser) : 

Modes 

9 

&  A 

188 

5.4 

4.5 

Optical 

(with  or  without 

laser) : 

Modes 

B 

&  C 

80 

1.5 

4.0 

As  Table  8  shows.  Fire  Duration  was  longest  for  the  Optical  Modes  at  10.0 
seconds,  next  longest  for  the  Radar  Pointer  Modes  at  7.3  seconds,  and  briefest 
for  the  Radar  Auto  mode,  at  5.2  seconds.  Frequently,  regardless  of  operation¬ 
al  mode,  the  weapon  system  automatically  ended  the  engagement.  Rather  than 
define  Fire  Duration  as  the  interval  from  trigger  pull  to  trigger  release,  it 
has  been  defined  as  the  interval  from  trigger  pull  to  computer  termination  of 
firing.  The  data  bus  contained  34  cases  where  no  data  was  given  for  trigger 
release,  116  cases  where  trigger  release  and  computer  termination  times  were 
Identical,  and  121  cases  where  the  trigger  release  time  was  greater  than  the 
computer  termination  time.  For  these  121  cases,  the  mean  trigger  release  time 
was  1.7  seconds  longer  than  the  computer  termination  time.  Over  271  engage¬ 
ments,  this  way  of  defining  Fire  Duration  shortened  the  mean  Fire  Duration  by 
0.8  seconds  compared  to  the  alternate  (and  expected)  way  of  defining  Fire 
Duration. 

The  Sgt  York  operator  manuals  recommended  that  firing  continue  for  3  to  7 
seconds.  Eighty-two  of  the  271  (-  30  %)  Fire  Duration  values  (measured  to 
computer  termination)  were  in  this  range.  Seventy-seven  of  the  237  (*  32%) 
Fire  Duration  values  (measured  to  trigger  release)  were  in  this  range. 


Table  8 


Modes  Comparison,  Nominal  System  Modes,  Fire  Duration 


ENGAGEMENTS  MEAN  S.D. 

(n) _ (s) _ (s) 


Radar  Auto:  Mode  8 

3 

5.2 

1.9 

Radar  Pointer 

(with  or  without  laser):  Modes  9  &  A 

188 

7.3 

7.3 

Optical 

(with  or  without  laser):  Modes  B  &  C 

80 

10.0 

9.2 
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Modes  Comparison,  Individual  Modes.  The  five  Individual  acquisition  modes 
were  compared  to  determine  if  there  were  any  differences  among  modes  for 
engagement  sequence  component  times.  The  five  system  modes  compared  were  (1) 
Mode  8,  Radar  Auto;  (2)  Mode  9,  Laser  Track,  Radar  Pointer;  (3)  Mode  A,  Radar 
Pointer;  (4)  Mode  B,  Laser  Track,  Radar,  Optical;  and  (5)  Mode  C,  Radar 
Optical.  For  these  five  modes,  statistical  significance  tests  were  possible 
on  only  six  of  the  eleven  components  of  the  target  engagement  sequence  because 
of  missing  variances  for  the  other  components.  The  individual  acquisition 
modes  were  significantly  different  on  all  six  of  the  comparisons  made:  (1) 
Target  Acquisition  (p<.001),  (2)  Fire  Solution:  Solid  (p<.001),  (3)  Fire 
Decision:  Solid  (p*.035),  (4)  Time  to  Fire  (p-.OOl),  (5)  Fire  Duration 
(p«.003),  and  (6)  Engagement  Duration  (p<.001).  Each  comparison  is  discussed 
in  turn  below. 

The  amount  of  time  it  took  for  the  weapon  system  to  acquire  the  target  was 
influenced  by  the  Individual  system  mode  being  used.  Acquisition  of  the 
target  was  actually  a  system  processing  function,  and  should  not  be  confused 
with  how  long  it  would  take  to  perform  crew  actions.  As  Table  9  shows,  in 
those  engagements  using  Optical  Mode  B  (a  mode  with  laser  available), 
acquisition  time  was  only  1.2  seconds,  while  Mode  C's  (the  Optical  Mode 
without  laser)  mean  acquisition  time  was  4.2  seconds.  Mode  9,  laser  track 
with  radar  pointer,  had  the  longest  acquisition  mean  time,  7.6  seconds. 


Table  9 

Modes  Comparison,  Individual  Modes,  Target  Acquisition 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Mode 

8, 

Radar 

Auto 

3 

5.2 

2.3 

Mode 

9, 

Laser 

Track,  Radar  Pointer 

26 

7.6 

6.9 

Mode 

A, 

Radar 

Pointer 

162 

5.0 

3.9 

Mode 

B, 

Laser 

Track,  Radar,  Optical 

72 

1.2 

3.4 

Mode 

c. 

Radar , 

,  Optical 

8 

4.2 

7.2 

As  indicated  in  Table  10,  the  time  required  for  the  weapon  system  to 
generate  a  solid  cue  fire  solution  varied  significantly  among  the  five 
individual  modes.  Fire  solutions,  whether  indicated  by  solid  or  blinking 
cues,  represented  a  weapon  system  processing  function  carried  out  by  the  fire 
solution  computer.  When  in  Optical  Mode  C  and  Radar  Auto  Mode  8,  the  weapon 
system  achieved  the  shortest  time  durations  for  system  processing  functions 
which  culminated  in  a  fire  solution  solid  fire  cue.  Again,  this  does  not 
represent  crew  action  or  crew  decision  times.  As  with  acquisition  time.  Mode 
9,  radar  pointer  with  laser  track,  had  the  longest  mean  time. 
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Table  10 


Modes  Comparison,  Individual  Modes,  Fire  Solution  Solid  Cue 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Mode 

8, 

Radar  Auto 

3 

0.6 

0.9 

Mode 

9, 

Laser  Track,  Radar  Pointer 

26 

5.3 

6.7 

Mode 

A, 

Radar  Pointer 

162 

1.9 

2.7 

Mode 

B, 

Laser  Track,  Radar,  Optical 

72 

3.4 

1.9 

Mode 

c. 

Radar,  Optical 

8 

0.9 

1.4 

Differences  In  crew  mean  fire  decision  times  following  a  solid  fire  cue 
are  shown  in  Table  11.  Fire  decision  time,  solid  cue,  is  defined  as  the 
amount  of  time  that  it  takes  a  crew  member  to  pull  the  trigger  after  a  solid 
fire  cue  appears  on  the  display.  Again,  Modes  C  and  8  had  the  shortest  mean 
time  durations.  Mode  9,  radar  pointer  with  laser,  required  the  longest  fire 
decision  time  of  the  five  modes  following  a  solid  fire  cue. 


Table  11 

Modes  Comparison,  Individual  Modes,  Fire  Decision  Following  Solid  Cue 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Mode 

8, 

Radar  Auto 

3 

0.4 

0.5 

Mode 

9, 

Laser  Track,  Radar  Pointer 

26 

3.2 

7.4 

Mode 

A, 

Radar  Pointer 

162 

1.2 

2.0 

Mode 

B, 

Laser  Track,  Radar,  Optical 

72 

1.0 

1.0 

Mode 

c, 

Radar,  Optical 

8 

0.2 

0.6 

Table  12  indicates  that  mean  Time  to  Fire  varied  significantly  (p».001) 
among  the  five  system  modes.  Time  to  Fire  was  a  combined  man-machine  system 
function  since  it  covered  the  target  engagement  sequence  from  the  time  of 
target  appearance  until  the  time  a  crew  member  depressed  the  trigger.  Thus, 
Time  to  Fire  incorporated  the  first  five  intervals  of  the  target  engagement 
sequence  (Select/Classify,  Point/Lase,  Acquisition,  Fire  Solution,  Fire 
Decision).  The  shortest  mean  Time  to  Fire  was  Mode  8's  6.7  seconds,  based  on 
only  three  cases.  Next  came  Mode  C,  with  9.5  seconds.  Mode  9  had  the  longest 
mean  Time  to  Fire,  24.6  seconds. 
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Table  12 


Modes  Comparison,  Individual  Modes,  Time  to  Fire 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(8) 

Mode 

8, 

Radar 

Auto 

3 

6.7 

2.4 

Mode 

9, 

Laser 

Track,  Radar  Pointer 

26 

24.6 

14.4 

Mode 

A, 

Radar 

Pointer 

162 

16.1 

9.9 

Mode 

B, 

Laser 

Track,  Radar,  Optical 

72 

15.7 

14.1 

Mode 

c, 

Radar 

,  Optical 

8 

9.5 

7.3 

Table  13  indicates  that  mean  Fire  Duration  varied  significantly  (p  ■*  .003) 
among  the  five  individual  modes.  Mode  8,  Radar  Auto,  had  the  shortest  Fire 
Duration  time,  5.2  seconds,  and  Mode  9  the  longest,  11.0  seconds.  Fire 
Duration  may  be  considered  a  crew  action  time  in  most  Instances.  It  is 
defined  for  the  purpose  of  this  report  as  the  amount  of  time  the  firing 
continued  for  each  engagement.  Specifically,  Fire  Duration  was  measured  by 
the  time  interval  from  the  beginning  of  the  trigger  pull  to  the  end  of  the 
engagement.  The  engagement  was  considered  terminated  when  the  gunner  released 
the  trigger,  or  when  the  weapon  system  ended  the  engagement,  usually  when 
radar  lock  was  lost. 


Table  13 

Modes  Comparison,  Individual  Modes,  Fire  Duration 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Mode 

8, 

Radar  Auto 

3 

5.2 

1.9 

Mode 

9, 

Laser  Track,  Radar  Pointer 

26 

11.0 

12.4 

Mode 

A, 

Radar  Pointer 

162 

6.6 

5.9 

Mode 

B, 

Laser  Track,  Radar,  Optical 

72 

10.4 

8.4 

Mode 

c. 

Radar,  Optical 

8 

6.7 

5.8 

Table  14  shows  the  results  of  comparing  mean  Engagement  Duration  among  the 
five  individual  modes.  Mean  Engagement  Duration  is  the  total  length  of  the 
target  engagement  from  the  appearance  of  the  target  to  the  cessation  of 
firing.  Differences  in  the  Total  Engagement  Duration  time  were  found  to  be 
significant  across  the  five  system  modes. 
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Table  14 


Modes  Comparison,  Individual  Modes,  Engagement  Duration 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(s) 

(8) 

Mode 

8, 

Radar  Auto 

3 

11.9 

3.9 

Mode 

9, 

Laser  Track,  Radar  Pointer 

26 

35.7 

21.5 

Mode 

A, 

Radar  Pointer 

162 

22.8 

11.7 

Mode 

B, 

Laser  Track,  Radar,  Optical 

72 

26.1 

15.2 

Mode 

c. 

Radar,  Optical 

8 

16.2 

7.6 

As  discussed  in  an  earlier  chapter,  the  Total  Engagement  Duration  Included 
three  components  associated  with  system  processing.  These  components  are  Ac¬ 
quisition  of  the  Target,  and  Fire  Solution  by  the  Computer  (Blinking  Cue  and 
Solid  Cue).  A  fourth  component.  Fire  Duration  Time,  might  sometimes  have 
Incorporated  an  aspect  of  system  processing,  for  engagements  were  terminated 
by  the  weapon  system  in  some  Instances. 

The  shortest  mean  Total  Engagement  Duration  was  associated  with  the  system 
mode  that  had  the  highest  level  of  automation.  Mode  8,  Radar  Auto,  with  a  mean 
of  11.9  seconds;  but  with  only  three  cases,  the  reliability  of  this  mean  is 
somewhat  suspect.  Another  finding  appears  relevant  to  the  concept  of  levels 
of  automation.  Mode  9,  which  included  both  the  Laser  Tracking  and  Radar 
Pointing  functions,  had  a  longer  Total  Engagement  Duration  than  the  other 
system  modes.  The  mean  engagement  time  for  Mode  9  was  35.7  seconds,  almost 
three  times  the  mean  for  Mode  8.  In  Mode  9,  the  crews  are  required  to  perform 
a  greater  number  of  procedures  in  order  to  activate  the  radar  pointer  and  arm 
and  point  the  laser.  Even  though  the  Radar  Auto,  Mode  8,  appeared  to  be  the 
most  viable  system  mode  to  use  for  reducing  Engagement  Duration  time,  the 
combination  modes  using  pointer  and/or  laser  were  the  more  frequently  employed 
modes  of  operation  during  the  Force-on-Force  phase  of  FOE  I. 

Modes  Comparison,  With  and  Without  Laser.  To  evaluate  the  impact  of  the 
laser,  another  modes  comparison  was  carried out .  Based  on  the  use  of  the 
laser,  engagements  were  configured  into  two  groups.  One  group  represented 
system  modes  where  the  crewmembers  used  the  laser;  the  other  group  represented 
system  modes  where  no  laser  was  used.  Summary  statistics  for  Modes  Com¬ 
parison,  With  or  Without  Laser,  were  analyzed.  Laser  versus  non-laser  system 
modes  of  operation  were  found  to  be  significantly  different  for  Acquisition, 
Fire  Solution  Solid,  Fire  Duration,  and  Engagement  Duration.  Conversely,  the 
two  groups  were  not  significantly  different  with  regard  to  Target  Selec¬ 
tion/Classification,  Pointing/Lasing,  Fire  Decision,  and  Time  to  Fire. 

The  acquisition  interval  was  the  time  between  when  the  target  had  been 
pointed  and  when  there  was  an  indication  that  a  laser  or  radar  cue  was  on  the 
display.  In  the  system  modes  where  lasing  had  been  enabled,  laser  data  could 
be  added  to  the  track  radar  data  to  establish  a  firing  solution.  Where  no 
radar  was  used  during  the  target  engagement,  lasing  provided  target  range  and 
speed  data  to  the  fire  control  computer.  When  the  laser  was  armed  and  the 
target  was  tracked  by  the  laser,  a  laser  cue  would  be  indicated  on  the 
display.  For  target  engagements  where  no  laser  was  used,  a  radar  cue  would  be 
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indicated  on  the  display  as  soon  as  the  tracking  radar  locked  on  the  target. 
When  system  modes  usiag  both  radar  and  laser  were  combined,  both  radar  and 
laser  cues  would  be  indicated  on  the  display  when  tracking  commenced. 

The  time  to  acquire  a  target  was  significantly  shorter  for  laser  than  for 
non-laser  modes  (p*.001).  As  Table  15  shows,  the  mean  acquisition  time  for 
lasing  modes  was  2.9  seconds;  it  was  5.0  seconds  for  the  non-lasing  modes, 
more  than  2  seconds  longer. 


Table  15 


Modes  Comparison,  With  or  Without  Laser,  Acquisition 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(8) 

With  Laser 

98 

2.9 

5.4 

Non-Laser 

173 

5.0 

3.6 

Fire  solution  with  a  solid  cue  was  a  system  processing  function  requiring 
information  on  range,  elevation,  and  azimuth.  Fire  solution  solid  cue  time  is 
defined  as  the  interval  between  the  appearance  of  the  radar  cue  or  the  laser 
cue,  and  the  subsequent  appearance  of  the  firing  cue.  It  represents  the  time 
it  took  the  fire  control  computer  to  obtain  a  solution.  Table  16  indicates 
that  the  mean  time  to  produce  a  solid  cue  fire  solution  differed  significantly 
(p  C.001)  from  lasing  modes  to  non-lasing  modes.  The  solid  firing  cue  was 
produced  2.0  seconds  faster  when  non-laser  modes  were  used  than  when  laser 
modes  were  used.  This  finding  contrasts  with  that  for  the  acquisition  system 
processing  time,  where  the  laser  mode  was  2.0  seconds  faster  than  the  non¬ 
laser  mode.  The  acquisition  time  gained  by  using  the  laser  was  lost  by  the 
longer  fire  solution  solid  cue  system  processing  time  for  laser  modes. 


Table  16 

Modes  Comparison,  With  or  Without  Laser,  Fire  Solution  Solid 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(s) 

With  Laser 

98 

3.9 

3.8 

Non-Laser 

173 

1.9 

2.7 

Table  17  indicates  that  Fire  Duration  varied  significantly  between  lasing 
and  non-lasing  modes  (p  <.001).  The  mean  Fire  Duration  time  for  non-laser 
modes  (6.6  secs)  was  within  the  3  to  7  seconds  mandated  by  the  Sgt  York 
operator  manuals.  Since  the  mean  Fire  Duration  for  laser  modes,  at  10.6 
seconds,  was  well  above  the  prescribed  time  period,  a  smaller  proportion  of 
the  laser  modes'  Fire  Durations  fell  within  the  3  to  7  seconds  range.  No 
reason  for  the  difference  in  Fire  Duration  means  is  readily  discernable. 
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Table  17 


Modes  Comparison,  With  or  Without  Laser,  Fire  Duration 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(8) 

With  Laser 

98 

10.6 

10.2 

Non-Laser 

173 

6.6 

5.9 

Laser  and  non-laser  acquisition  modes  were  also  significantly  different 
(p<.001)  with  respect  to  Total  Engagement  Duration  time,  as  is  shown  in  Table 
18.  Laser  engagements'  mean  duration  was  6.3  seconds  longer  than  non-laser 
engagements,  but  the  blame  cannot  be  put  on  the  use  of  the  laser  itself;  for, 
as  Table  4  indicates,  the  Laser  On  time  was  but  1.1  seconds  compared  to  the 
Pointer  On  time  of  2.3  seconds.  Other  components,  such  as  Fire  Duration,  are 
responsible  for  the  longer  Laser  Total  Engagement  Duration. 


Table  18 

Modes  Comparison,  With  or  Without  Laser,  Engagement  Duration 


ENGAGEMENTS 

MEAN 

(n) 

(8) 

(s) 

With  Laser 

98 

28.6 

17.5 

Non-Laser 

173 

22.3 

11.6 

Levels  of  Automation  Comparisons.  As  has  been  observed  throughout  this 
section,  the  times  associated  with  the  Radar  Auto  mode,  Mode  8,  show  that 
engagements  conducted  using  this  mode  were  briefer  than  engagements  in  which 
other  modes  were  used.  The  Radar  Auto  mode  represented  the  highest  level  of 
automation  available  with  the  Sgt  York  system.  It  should  be  remembered  that 
in  all  engagements  the  fire  decision  (implemented  by  pulling  the  trigger)  was 
to  be  made  by  one  of  the  crew  members,  not  by  the  automatic  system. 

A  close  inspection  of  the  1553  data  bus  record  revealed  that  the  crew 
members'  decision  role  was  preempted  by  the  automatic  system  in  two  ways  on 
some  engagements.  One  type  of  preemption  concerned  the  firing  decision.  On 
some  occasions  while  the  trigger  was  being  pressed  and  a  target  was  being 
fired  on,  the  target  being  engaged  would  change  without  any  action  by  the  crew 
members  to  occasion  such  a  change.  If  the  gunner  was  not  extremely  alert,  he 
would  continue  firing  but  on  a  different  target  than  the  one  he  had  selected. 
When  the  system  switched  targets.  Fire  Duration  for  the  operator-selected 
target  was  terminated  automatically  (and  perhaps  prematurely).  Often,  after  a 
brief  trigger  release  for  confirmation  of  the  foe  classification  of  the  new 
target,  the  gunner  would  reengage  the  new  target  and  fire  purposefully  on  the 
new  target. 

Another  preemption  by  the  system  occurred  earlier  in  the  target  engagement 
sequence.  Sometimes  without  any  recorded  crew  action  to  select  a  target, 
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whether  by  pointing  or  lasing,  a  target  would  suddenly  be  engaged  by  the 
system.  This  action  was  clearly  Indicated  on  the  1553  record  by  a  notation 
showing  "target  number"  under  the  1553  column  headed  target  engaged.  Both  of 
these  preemptions  share  the  characteristic  that  the  system  has  established  a 
radar  or  laser  cue  relating  to  a  new  target.  The  next  system  step  will  be  to 
determine  a  firing  solution.  These  steps  also  occur  when  the  system  is  in  the 
Radar  Auto  Mode.  System  preempts  did  not  exhibit  the  Radar  Auto  property  of 
controlling  the  gun  turret's  movement. 

To  ascertain  what  effect  such  system-initiated  engagements  might  have  had 
on  the  times  for  the  various  segments  of  the  target  engagement  sequence, 
another  comparison  was  carried  out  with  the  "system-preempts”  forming  a 
separate  category,  or  after-the-fact  mode.  Information  would  be  sought  on  how 
similar  the  system  preempt  mean  intervals  would  be  to  the  Radar  Auto  mode  and 
the  other  modes  as  well.  Radar  Auto  engagements  were  considered  the  highest 
level  of  automation,  able  to  perform  the  engagement  functions  most  quickly  of 
the  designed  modes.  The  four  categories  to  be  compared  were: 


(1)  Radar  Auto, 

(2)  All  system  preempt  engagements  (coded  in  Table  3  as  **  engagements), 
whatever  the  nominal  mode  appearing  on  the  data  bus, 

(3)  All  engagements  in  which  radar  pointing  was  the  primary  way  used 
to  select  a  target,  and 

(4)  All  engagements  in  which  the  laser  was  used  as  the  primary  way  of 
selecting  a  target. 

As  Indicated  in  Table  19,  these  four  levels  of  automation  were  found  to  be 
significantly  different  (p<.001)  with  respect  to  four  parts  of  the  target 
engagement  sequence:  Target  Acquisition,  Fire  Solution,  Time  to  Fire,  and 
Total  Engagement  Duration. 

Table  19  shows  system  Target  Acquisition  time  as  a  function  of  automation 
level.  In  comparing  acquisition  times,  it  must  be  remembered  that,  for  both 
Radar  Auto  engagements  and  system-preempt  engagements,  no  crew  action  preceded 
acquisition,  so  the  5.2  seconds  for  acquisition  in  the  Radar  Auto  mode  and 
the  6.6  seconds  for  acquisition  during  system  preempt  engagements  represents 
the  total  time  for  the  engagement  up  to  the  onset  of  the  radar  or  laser  cue. 
For  Radar  Pointer  engagements,  another  10.0  seconds  of  crew  decision  and 
action  time  had  preceded  the  4.9  seconds  attributable  to  acquisition,  bringing 
the  Radar  Pointer  total  time  to  acquisition  to  14.9  seconds  (for  which  see 
Table  20).  For  Laser  engagements,  9.4  seconds  of  crew  decision  and  action 
time  had  preceded  the  0.10  acquisition  time,  bringing  the  Laser  engagement 
total  time  to  acquisition  to  9.5  seconds. 


92 


Table  19 


Levels  of  Automation  Comparisons,  Acquisition 


ENGAGEMENTS 

(n) 

MEAN 

(8) 

S.D. 

(s) 

Radar  Auto 

3 

5.2 

2.3 

System  Preempts 

43 

6.6 

6.4 

Radar  Pointer  Primary 

155 

4.9 

3.3 

Lase  Primary/ Optical 

66 

0.10 

0.07 

Table  20  shows  the  mean  total  time  It  took  to  achieve  Target  Acquisition 
as  a  function  of  level  of  automation.  Again,  this  is  the  interval  from  target 
appearance  to  radar  or  laser  cue  onset.  No  radar/ laser  buttons  are  depressed 
or  released  by  the  computer  for  the  Radar  Auto  or  System  Preempts;  so,  for 
these  two  categories,  the  first  three  time  intervals  of  the  engagement 
sequence  are  not  measurable  -  or  one  may  consider  their  values  to  equal  zero. 

A  new  interval,  called  Total  Time  to  Acquisition,  is  defined  and  shown  in 
Table  20.  It  appears  that  the  system  preempts  level  of  automation  falls 
between  Radar  Auto  engagements  and  crew-initiated  engagements  as  regards  Total 
Time  to  Acquisition. 


Table  20 


Levels  of  Automation  Comparisons,  Total  Time  to  Acquisition 

Level  Mean  Total  Time  (s) 


Radar  Auto  5.2 
System  Preempts  6.6 
Radar  Pointer  Primary  14.9 
Lase  Primary/ Optical  9.5 


The  mean  times  for  fire  solution  solid  as  a  function  of  the  defined  levels 
of  automation  are  shown  in  Table  21.  The  shortest  mean  times  of  0.6  seconds 
were  found  for  both  the  system-preempt  engagements  and  Radar  Auto  fire 
engagements.  Mean  times  for  the  other  two  categories.  Radar  Pointer  and  Laser 
engagements,  are  more  than  2  seconds  longer.  Perhaps  some  unapparent  parallel 
processing  was  possible  in  the  first  two  types  of  engagements  but  not  in  the 
latter  two.  On  thiB  variable,  system  preempts  perform  at  the  same  level  as 
Radar  Auto. 
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Table  21 


Levels  of  Automation  Comparisons,  Fire  Solution  Solid 


ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Radar  Auto 

3 

0.6 

0.9 

System  Preempts 

43 

0.6 

1.7 

Radar  Pointer  Primary 

155 

2.8 

3.7 

Lase  Primary/Optical 

50 

3.5 

1.8 

Table  22  shows  the  mean  Time  to  Fire  for  the  four  levels  of  automation 
comparison.  The  mean  for  the  three  engagements  In  the  Radar  Auto  mode  was  6.7 
seconds.  The  individual  engagement  values  were  4.5,  6.5,  and  9.2  seconds, 
meaning  that  two  of  the  three  times  were  under  7  seconds.  The  mean  Time  to 
Fire  for  each  of  the  other  three  categories  of  engagements  exceeded  7  seconds. 
The  mean  for  the  43  system  preempt  engagements  exceeded  it  by  0.9  seconds. 
Examination  of  the  individual  times  revealed  that  51%  (22  of  43)  System 
Preempt  Time  to  Fire  scores  were  under  7  seconds.  The  mean  Time  to  Fire  for 
the  66  Laser  and/or  optical  engagements  was  14.5  seconds,  exceeding  the  system 
operational  requirement  by  7.5  seconds.  Surprisingly,  perhaps,  46%  of  50 
examined  laser  engagements  took  less  than  seven  seconds.  The  mean  Time  to 
Fire  for  the  155  Radar  Pointer  engagements  was  19.8  seconds,  12.8  seconds  over 
the  requirement.  Only  2%  (3  of  155)  of  the  individual  scores  were  less  than 
seven  seconds. 


Table  22 

Levels  of  Automation  Comparisons,  Time  to  Fire 


ENGAGEMENTS 

(n) 

MEAN 

(8) 

S.D. 

(s) 

Radar  Auto 

3 

6.7 

2.4 

System  Preempts 

43 

7.9 

6.9 

Radar  Pointer  Primary 

155 

19.8 

11.1 

Lase  Primary/Optical 

66 

14.5 

13.0 

As  Table  23  shows,  the  relationship  among  the  four  mean  Total  Engagement 
Duration  times  was  similar  to  the  relationship  for  Time  to  Fire  means  in  that 
the  Radar  Auto  mean  was  shortest  and  Radar  Pointer  mean  was  longest.  Laser 
engagements  were  again  shorter  than  Radar  Pointer  ones,  but  whether  that 
advantage  was  related  to  previous  operator  experience  with  laser  systems  or  to 
some  aspect  of  the  Sgt  York  system  is  unclear.  The  system  preempts  mean  is 
close  to  the  Radar  Auto  mean,  as  it  was  for  the  Time  to  Fire  variable. 
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Table  23 

Levels  of  Automation  Comparisons,  Engagement  Duration 


ENGAGEMENTS 

(n) 

MEAN 

(6) 

S.D. 

(8) 

Radar  Auto 

3 

11.9 

3.9 

System  Preempts 

43 

14.8 

8.2 

Radar  Pointer  Primary 

155 

14.0 

Lase  Primary/Optical 

66 

CD 

• 

CM 

14.5 

One  other  comparison  Is  worthy  of  comment.  Radar  Auto  engagements  and 
system-initiated  engagements  were  compared  with  gunner-initiated  engagements 
and  squad  leader-initiated  engagements.  The  breakdown  used  earlier  in  Figure 
21  was  used  for  this  comparison.  As  noted  earlier,  only  the  mean  for  the 
Radar  Auto  engagements  is  less  than  seven  seconds.  System  initiated  engage¬ 
ments  were  next  fastest,  but  gunner-initiated  engagements  were  more  than  2 
seconds  faster,  with  respect  to  Time  to  Fire,  than  were  squad  leader- initiated 
engagements.  Since  squad  leader-initiated  engagements  required  coordination 
between  the  two  crew  members  and  gunner-initiated  ones  did  not,  this  coordina¬ 
tion  time  may  account  for  the  2.4  seconds  shorter  mean  Time  to  Fire  taken  by 
the  gunners. 


Table  24 

System  and  Subsystem  Performing  Times  as  a  Function  of  Engagement 
Initiator 


Initiator 

Radar 

System 

Gunner 

Squad  Leader 

Auto 

Initiated 

Initiated 

Initiated 

Subystem  Component 

(s) 

(8) 

(8) 

(8) 

Crew  Target  Analysis 

0 

0 

7.6 

8.4 

Crew  Target  Action 

0 

0 

1.8 

2.2 

Automated  Decision  Support 

6.1 

6.6 

6.8 

7.3 

Crew  Decision  to  Fire 

0.7 

0.7 

1.3 

1.9 

Time  to  Fire 

6.7 

7.9 

17.3 

19.7 

In  Figure  22,  the  target  engagement  sequence  is  broken  down  into  five 
components:  crew  target  analysis,  crew  target  action,  system  target  analysis, 
automated  decision  support,  and  crew  decision  to  fire.  Figure  22  parallels 
data  in  Table  24,  except  that  what  is  shown  as  "automated  decision  support”  in 
Table  24  is  further  droken  down  into  "system  target  analysis"  and  "automated 
decision  support”  in  the  figure.  Vertical  bars  are  included  in  Figure  22  to 
represent  the  four  levels  of  automation  discussed  here,  as  well  as  for  gunner 
initiated  and  squad  leader-initiated  engagements.  As  the  figure  makes 
apparent,  integrating  human  operators  into  the  target  engagment  sequence  in 
the  case  of  the  Sgt  York  not  only  added  additional  steps  to  the  sequence  (crew 
action  and  crew  decision  steps) ,  but  lengthened  the  time  required  to  perform 
the  remaining  steps,  the  steps  common  to  automated  and  crew-involved  engage- 
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meats.  That  is,  if  man  is  to  be  in  the  loop,  his  influence  may  extend  beyond 
the  steps  he  performs  and  in  this  system  may  have  lengthened  the  other  steps 
as  well. 

Fire  Unit  Comparisons 

Five  Sgt  York  fire  units  were  involved  in  the  FOE  I  Force-on-Force  trials. 
Times  for  each  component  of  the  target  engagement  sequence  were  broken  out 
separately  by  fire  unit  in  order  to  identify  any  differences  that  might  exist 
between  fire  units.  Note,  however,  that  crew  identity  is  extensively 
confounded  with  fire  unit  Identity.  As  Table  25  shows,  a  given  crew  (l.e.,  a 
specific  squad  leader  paired  with  a  specific  gunner)  usually  stayed  with  a 
given  fire  unit.  Fire  Units  1  and  2  had  no  crew  changes  for  the  trials 
analyzed.  Fire  Unit  4  had  the  same  crew  for  trials  1-14,  and  a  completely 
different  crew  for  trial  15.  Fire  Units  3  and  5  had  the  same  squad  leader 
throughout,  but  changed  gunners  as  indicated.  For  this  analysis,  only  fire 
unit  identity  is  specified,  despite  its  confounding  with  crew  identity.  Crew 
composition  will  be  treated  later  in  this  chapter. 


96 


Time  (Seconds) 


System  Target  Analysis 


Table  25 

Crews  and  Fire  Units  by  Trial 


Trial 

Fire  Fire 

Fire 

Fire 

Fire 

Number 

Unit 

:  1  Unit  2 

Unit  3 

Unit  4 

Unit  5 

1020 

A 

E 

C 

B 

1021 

D 

H 

C 

B 

1022 

D 

H 

C 

B 

1023 

A 

D 

G 

C 

1026 

A 

E 

C 

B 

1027 

A 

E 

C 

B 

1029 

A 

D 

C 

B 

1035 

A 

D 

G 

C 

1036 

A 

D 

G 

C 

1039 

A 

D 

G 

C 

1040 

A 

D 

G 

C 

1041 

A 

D 

F 

B 

1042 

A 

D 

C 

B 

1046 

A 

D 

G 

I 

1048 

A 

F 

D 

B 

Key: 

Crew 

Squad  Leader  ID  Gunner 

ID 

A 

9089 

6111 

B 

4343 

0240 

C 

1746 

9843 

D 

6640 

5197 

E 

3753 

5197 

F 

3753 

9843 

G 

3753 

0240 

H 

3753 

6111 

I 

4343 

9843 

Table  26  presents  a  comparison  of  the  target  engagement  sequence  times 

the  different  fire  units.  Fire 

Unit  5  had 

shorter  mean 

times  than  did  the 

other  fire 

units 

for  both  of  the 

summary  measures,  e.g. 

,  Time  to  Fire  and 

Total  Engagement 

Duration.  Fire 

Unit  5  began  firing  more  than  4.5  seconds 

sooner  than 

Fire 

Unit  4,  the  second  fastest 

fire  unit. 

The  Time  to  Fire 

differences 

were 

significant  (p* 

.005). 

Fire  Unit  5  also  had  the  shortest  mean  Total  Engagement  Duration.  The 
difference  was  not  as  great  as  the  Time  to  Fire  difference,  since  Engagement 
Duration  Included  Fire  Duration  and  the  mean  Fire  Duration  for  Fire  Unit  5 
was  relatively  long  (8.6  seconds).  The  differences  in  Fire  Durations  were 
not  significant,  but  only  Fire  Unit  2  had  a  longer  mean  Fire  Duration. 
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Table  26 


Mean  Times  by  Fire  Unit  for  Target  Engagement  Components 


FU1 

Mean 

FU2 

i  Times 
FU3 

(s) 

FU4 

FU5 

Significance 

Level 

Select 

12.1 

8.1 

8.1 

7.9 

4.7 

.040 

Pointer  On 

1.6 

3.5 

2.1 

2.5 

1.5 

.005 

Laser  On 

0.8 

1.3 

1.0 

0.7 

1.5 

NS 

Acquisition 

3.6 

4.5 

5.0 

4.2 

3.7 

NS 

Fire  Solution 
Blinking 

5.3 

6.1 

2.3 

4.2 

2.4 

.005 

Fire  Solution 
Solid 

2.6 

2.8 

3.0 

2.6 

2.3 

NS 

Fire  Decision 
Blinking 

1.2 

3.3 

1.2 

1.4 

1.7 

NS 

Fire  Decision 
Solid 

0.8 

0.8 

1.8 

1.2 

1.3 

NS 

Time  to  Fire 

19.7 

19.4 

17.5 

16.7 

12.2 

.005 

Fire  Duration 

8.5 

10.4 

6.8 

7.1 

8.6 

NS 

Engagement  Duratlon28.2 

29.8 

24.4 

23.8 

20.8 

.014 

Despite  its  relatively  long  Fire  Duration  mean.  Fire  Unit  5  still  had  a  mean 
Engagement  Duration  more  than  3  seconds  shorter  than  Fire  Unit  4,  the  next 
fastest  engagement  mean.  Engagement  Duration  differences  were  significant 
(p-,014). 

Times  for  the  remaining  eight  components  are  also  presented  in  Table  26. 

On  five  of  the  eight  components,  there  were  no  significant  differences  among 
the  fire  units.  However,  for  Select  Time,  Pointer  on  Time,  and  Fire  Solution 
Blinking  time,  there  were  significant  differences.  On  the  first  two,  both 
involving  crew  functions,  Fire  Unit  5  was  the  fastest.  On  the  third,  a  system 
processing  time.  Fire  Unit  5  was  the  second  fastest,  but  it  was  only  0.1 
slower  than  the  fastest  Fire  Unit,  unit  3.  There  was  a  spread  of  3.8  seconds 
from  the  fastest  to  the  slowest  Fire  Solution  blinking  time  (2.3  to  6.1 
seconds).  The  spread  of  Fire  Solution  Solid  mean  times  was  much  narrower, 

0.7  seconds  (2.3  to  3.0  seconds),  and  those  differences  were  not  significant. 

In  summary,  it  appears  that  the  shorter  overall  times  achieved  by  Fire 
Unit  5  represented  both  quicker  crew  response  times  (Select  and  Pointer  On 
times)  and  quicker  hardware-software  function  (Acquisition  and  Fire  Solution 
times).  However,  the  system  processing  times  for  Fire  Unit  5  were  not 
significantly  shorter;  so  it  does  not  appear  that  Fire  Unit  5  was  faster  or 
consistently  different  from  the  other  fire  units. 
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Tactical  Performance  Variations 


Forward  area  air  defense  offers  a  very  rich  environment  for  performance 
variations  as  a  function  of  different  scenario/battle  conditions.  Data  are 
provided  here  for  the  following  variables:  one.  Scenario  type;  two,  Rotary¬ 
wing  tactics;  three,  ECM  condition;  four.  Time  of  day;  five,  Target  range  at 
first  appearance. 

Scenario  Type 


Scenario  Descriptions.  Three  different  scenarios  were  used  for  the 
Force-on-Force  trials:  (1)  Delay,  (2)  Attack,  and  (3)  Road  March.  Each 
scenario  differed  in  terms  of  the  tactical  mission  assigned  to  the  Blue  Force. 
In  addition  to  the  Blue  force  which  portrayed  the  friendly  force,  there  was 
the  Red  Force  which  simulated  the  enemy.  Two  versions  of  each  scenario  type 
were  implemented. 

Delay  scenarios  had  the  Blue  Force  performing  a  defensive  mission.  Two 
terrain  sites  were  used  for  the  two  Delay  scenarios:  (1)  Gabilan  Valley  with 
the  Blue  Force  oriented  toward  the  southeast,  and  (2)  San  Mlguelito  Loop  with 
the  Blue  Force  oriented  toward  the  northwest.  Ten  valid  Sgt  York  trials  were 
run  using  delay  scenarios.  Six  of  them  were  analyzed,  three  of  them  with  the 
Gabilan  Valley  site  and  southeast  orientation  (1021,  1023,  and  1027)  and  three 
of  them  with  the  San  Miguelito  Loop  site  and  the  northwest  orientation  (1029, 
1035,  and  1042).  The  relationship  of  scenario  to  terrain  site  and  orientation 
is  shown  in  Table  27. 


Table  27 

Terrain  Site  and  Orientation 


Scenario  Type 

Gabilan 

Valley 

San  Miguel¬ 
ito  Loop 

Southeast 

Northwest 

Delay  (1) 

Delay  (2) 

X 

X 

X 

X 

Attack  (3) 

Attack  (4) 

X 

X 

X 

X 

Road  March  (5) 
Road  March  (6) 

X 

X 

X 

X 

Attack  scenarios  represented  the  Blue  Force  exercising  an  offensive 
mission.  There  were  14  valid  Attack  trials  and  7  of  them  were  analyzed  (1020, 
1022,  1036,  1039,  1041,  1046,  and  1048).  The  Attack  scenarios  used 
the  same  two  basic  terrain  sites:  (1)  San  Miguelito  Loop  with  the  Blue  Force 
oriented  toward  the  southeast,  and  (2)  Gabilan  Valley  with  the  Blue  Force 
oriented  toward  the  northwest. 

The  Road  March  scenario  consisted  of  the  Blue  Force  on  tactical  road 
march  moving  from  the  rear  toward  the  Forward  Edge  of  the  Battle  Area  (FEBA). 
Sgt  York  valid  trials  included  five  Road  March  scenarios;  two  of  the  them  were 
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analyzed  (1026  and  1040).  During  Che  Road  March,  Che  same  two  terrain  sites 
were  used:  (1)  San  Miguellto  Loop  with  the  Blue  Force  oriented  toward  the 
southeast,  and  (2)  Gabllan  Valley  with  the  Blue  Force  oriented  toward  the 
northwest.  Trial  1040  was  a  night  road  march  and  Trial  1054  was  planned  as  a 
second  night  road  march,  but  test  changes  were  made  during  its  execution.  The 
second  night  road  march  was  run  with  the  Blue  Force  (including  the  Fire  Units) 
deployed  along  the  march  route  prior  to  trial  start  time.  This  operational 
change  was  made  to  avoid  safety  problems.  There  was  concern  for  the  safety  of 
the  crews  if  they  had  to  manuever  the  Sgt  York  in  the  dark.  Because  of  this 
difference  with  Trial  1040,  this  trial  was  not  analyzed. 

Fifteen  trials  are  identified  in  Table  28  according  to  trial  number,  date 
conducted,  scenario  type,  scenario  number,  day/night  conditions,  end  of  trial 
time,  rotary  wing  tactics,  level  of  Mission  Oriented  Protective  Posture 
(MOPP) ,  Electronic  Countermeasures  (ECM)  condition,  replication,  and  order  in 
day.  The  term  "replication"  indicates  how  many  times  the  trial  was  initiated 
until  it  was  completed  successfully.  For  example.  Trial  1020  was  completed 
successfully  on  its  first  initiation  (this  is  noted  by  the  symbol  "A"  in  the 
replication  column).  Trial  1048  was  completed  successfully  on  its  third 
initiation  which  is  noted  by  a  “C."  The  second  field  is  termed  "Order  in  Day” 
and  indicates  whether  the  trial  was  being  run  as  the  first,  second,  or  third 
trial  of  the  day. 

Scenario  Comparisons.  To  analyze  the  effects  of  the  three  basic  scenario 
types,  the  two  Delay  scenarios  were  combined,  the  two  Attack  scenarios  were 
combined,  and  the  two  Road  March  scenarios  were  combined.  In  the  dBase  III 
data  base  created  for  this  study,  there  were  7  Attack  trials,  6  Delay  trials, 
and  2  Road  March  trials.  Within  the  29  valid  Sgt  York  FOE  I  trials,  there 
were  14  Attack  scenario  trials,  10  Delay  scenario  trials,  and  5  Road  March 
scenario  trials. 

Scenario  type  came  close  to  the  statistical  significance  criterion  for 
only  one  of  the  target  engagement  sequence  components ,  reaching  it  for  none . 
Mean  times  for  Fire  Solution  Solid  Cue  were  found  to  be  very  close  (p  **  .051) 
to  the  criterion  for  rejecting  the  hypothesis  of  no  difference  when  the  three 
scenario  types  were  compared  using  analysis  of  variance  (see  Table  29).  Did 
something  about  the  fire  unit's  moving  during  the  road  march  increase  the 
computer's  time  to  process  a  fire  solution? 

During  the  Road  March  condition,  it  did  take  longer  for  the  fire  control 
computer  to  obtain  a  solid  firing  cue.  The  Road  March  represents  firing  on 
the  move  (known  as  shoot  on  the  move  capability),  and  the  fire  units  were 
designed  for  such  conditions.  It  was  indicated  in  the  Operator's  Manual  that 
there  would  be  a  loss  of  accuracy  when  firing  on  the  move.  According  to  the 
operational  and  organization  plan,  Sgt  Yorks  were  to  provide  convoy/escort 
defense  for  maneuver  elements  during  road  marches  when  they  were  not  in 
contact  with  enemy  forces.  Even  though  Sgt  Yorks  had  shoot  on  the  move 
capability,  firing  on  the  move  was  considered  particularly  difficult  for  the 
gunner  (the  difficulty  associated  with  his  keeping  his  eye  in  the  gunsight). 

It  should  be  kept  in  mind  that  in  situations  requiring  a  tactical  road  march, 
the  purpose  of  the  road  march  is  for  relocation  and  not  for  enemy  contact. 
Tactical  road  marches  are  conducted  at  prescribed  rates  of  speed  with 
prescribed  intervals  between  vehicles.  Findings  for  scenario  differences, 
including  Road  March,  indicate  that  there  was  no  significant  difference  for 
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Scenarios  Carried  Out  During  Sgt  York  FOB  I  Tests,  Detailed  in  Order  of  Their  Occurrence 


Order 

in 

Day 

NHNrtNHHMCMNnHNNN 
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ECM 

Condition 

IOC 

IOC 

Benign 

IOC 

IOC 

IOC 

IOC 

IOC 

IOC 

Benign 

Design 

Design 

Benign 

Design 

Design 

MDPP 

Yes/ 

No 

No 

No 

No 

No 

No 

Yes 

Yes 

No 

No 

No 

No 

NO 

No 

No 

No 

Rotary  Wing 
Tactics 

Havoc /Hover 

Hind/Running 
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crew  decision  time  and  crew  action  time  among  scenario  conditions.  Even 
under  Road  March  conditions,  crews  were  able  to  maintain  approximate  levels  of 
functioning  for  target  engagement  sequences.  For  future  FAADS  designs, 
examining  performance  while  moving  would  appear  to  be  a  more  reasonable 
concern  even  at  the  expense  of  some  scenario  differences. 


Table  29 

Scenario  Comparisons  for  Time  for  Three  Scenario  Types  -  Fire  Solution  Solid 


Scenario  Type 

ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(8) 

Delay 

77 

2.0 

1.9 

Attack 

153 

2.8 

3.7 

Road  March 

19 

4.5 

1.6 

Table  30 


Scenario  Comparisons  for  Delay  1  &  2, 
Solution  Blinking 

Attack  1  &  2,  Road 

March  ] 

.  -  Fire 

Scenario  Type 

ENGAGEMENTS 

(n) 

MEAN 

(s) 

S.D. 

(s) 

Delay  1 

52 

2.9 

1.8 

Delay  2 

47 

6.0 

3.6 

Attack  3 

73 

2.5 

1.6 

Attack  4 

80 

4.4 

4.1 

Road  March  5 

19 

4.5 

1.6 

A  further  comparison  of  the  scenarios  was  performed.  As  described 
earlier,  the  performance  data  base  included  data  on  two  variations  of  the 
Delay  and  Attack  scenarios  and  one  variation  of  the  Road  March.  Thus,  there 
were  five  separate  scenario  subtypes  available  for  this  further  scenario 
comparison.  Analysis  of  variance  revealed  that  there  was  a  significant 
difference  among  these  five  scenarios  (p».023)  for  Fire  Solution  Blinking  (see 
Table  30).  In  scenario  Attack  3  and  scenario  Delay  1,  a  fire  solution 
blinking  cue  was  obtained  more  quickly  than  in  the  other  three  scenarios. 

These  two  scenarios  (Attack  3  and  Delay  1)  had  different  terrain  sites  and  a 
different  geographic  orientation  so  that  location  during  scenario  does  not 
appear  to  account  for  the  magnitude  of  their  blinking  fire  solution  cue  means. 
The  blinking  cue  does  not  reflect  crew  decision  time  or  crew  action  time, 
rather  it  is  a  weapon  system  processing  time  that  represents  the  fire  control 
computer's  time  to  calculate  a  fire  control  solution.  As  noted  earlier,  the 
blinking  cue  Indicated  a  less  certain  solution  than  a  solid  cue.  Why  there 
should  have  been  more  than  a  3-second  difference  between  Delay  (1)  and  Delay 
(2)  trials,  or  a  1.9-second  difference  between  Attack  (3)  and  Attack  (4) 
trials,  is  unclear. 


103 


In  this  five  scenario  comparison,  no  crew  decision  or  crew  action  times 
were  significantly  different  among  the  scenarios.  Apparently,  crew  decision 
and  action  times  were  comparable  within  the  scenario  variations. 

Rotary-Wing  Tactics 

One  of  the  variables  employed  during  the  Sgt  York  FOE  I  Force-on-Force 
trials  concerned  the  tactics  used  by  the  rotary-wing  aircraft  involved  in 
these  tests.  Two  different  hostile  helicopters,  the  Havoc  and  the  Hind,  were 
simulated  during  these  tests  by  AH-64s.  In  the  case  of  the  Havoc,  the  AH-64 
employed  a  hover  tactic.  In  the  case  of  the  Hind,  two  different  tactics  were 
used,  a  hover  tactic,  as  with  the  Havoc,  and  a  running  tactic.  Thus,  there 
were  three  helicopter/ tactic  mixes:  Havoc/Hover,  Hind/Hover,  and  Hind/Runn¬ 
ing.  One  of  these  three  mixes  was  assigned  to  each  Sgt  York  trial.  That  is, 
for  a  given  trial,  the  AH-64s  simulated  either  Havoc  or  Hind  helicopter;  if 
Havoc,  they  used  a  hover  tactic;  and  if  Hind,  either  a  hover  or  a  running 
tactic.  The  type  of  helicopter  simulated  did  not  vary  within  a  single  trial, 
nor  did  the  tactic  used  vary  within  the  trial. 

Table  31  shows  the  mean  times  for  all  components  of  the  target  engagement 
sequence,  presented  separately  for  the  three  rotary-wing  tactics  discussed 
above.  There  are  significant  differences  on  only  three  of  the  measures.  Laser 
On  Time,  Time  to  Fire,  and  Engagement  Duration.  For  all  three  of  these 
categories,  the  time  was  shortest  for  Hind/Running.  Both  Time  to  Fire  and 
total  engagement  time  suggest  that  the  Havoc/Hover  tactic  was  the  most 
difficult  of  the  three  presented  for  the  system  to  handle.  However,  laser  on 
time  was  longest  for  Hind/Hover. 


Table  31 

A  Comparison  of  Mean  Times  for  Target  Engagement  Sequence  Tasks  According  to 
Rotary-Wing  Tactic  Used 

' """  Tasks  Rotary-Wing  Tactic  Significance 

Level 

I  Havoc/  I  Hind/  |  Hind/  I 
Hover  Hover  Running 

_ (s) _ (s) _ (8) _ 


Select 

8.9 

Pointer  On 

2.1 

Laser  On 

0.9 

Acquisition 

4.7 

Fire  Solution: 

Blinking 

4.0 

Fire  Solution: 

Solid 

2.8 

Fire  Decision: 

Blink! ng 

1.5 

Fire  Decision: 

Solid 

1.5 

Time  to  Fire 

18.1 

Fire  Duration 

8.4 

Engagement  Time 

26.4 

6.0 

7.3 

NS 

2.4 

2.7 

NS 

1.6 

0.7 

.007 

3.5 

3.8 

NS 

4.5 

3.5 

NS 

2.6 

1.4 

NS 

2.0 

1.8 

NS 

0.8 

1.1 

NS 

14.6 

14.0 

7.9 

7.1 

NS 

22.5 

21.1 
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It  is  tempting  to  try  to  derive  hypotheses  about  the  effect  of  tactics  on 
operator  and  machine  performance  by  noting  the  differences  in  mean  times 
entered  in  Table  31.  The  FOE  trials  analyzed  here  do  not  resolve  the  question 
of  whether  or  not  the  tactics  used  by  enemy  aircraft  affect  the  speed  with 
which  human  operators  handle  such  targets  or  influence  the  speed  of  the 
hardware/ software  system's  operations.  There  are  differences,  but  since  they 
did  not  reach  the  designated  level  of  significance,  they  should  not  be  con¬ 
sidered  real  or  reliable  differences. 

An  additional  factor  should  be  considered  in  comparing  the  times  as¬ 
sociated  with  the  three  rotary-wing  tactic  conditions.  As  noted  earlier  in 
this  chapter  in  the  discussion  of  levels  of  automation  and  system  modes, 
engagements  which  were  initiated  independent  of  crew  action,  including  those 
run  in  the  Radar  Auto  mode,  had  shorter  Time  to  Fire  times  and  total  engage¬ 
ment  times  than  did  the  other  engagements.  Of  the  engagements  run  in  the 
Hind/ Running  condition,  which  had  the  shortest  Time  to  Fire  and  Engagement 
Times  of  the  three  tactical  conditions,  702  were  initiated  by  crew  members. 

In  the  Hind/Hover  condition,  872  were  crew  initiated,  and  in  the  Havoc/Hover, 
822.  So  the  system-preempt  engagements  contributed  their  quickness  more  to 
the  Hind/Running  condition  than  to  the  other  two  helicopter-tactic  mixes,  thus 
apparently  contaminating  the  comparison.  Whether  or  not  the  relationship 
between  tactical  condition  and  proportion  of  crew-initiated  engagements  is 
anything  other  than  chance  remains  an  open  question. 

ECM  Conditions 


The  Sgt  York  FOE  I  Force-on-Force  trials  were  run  under  three  different 
electronic  countermeasure  (ECM)  conditions.  That  1b,  each  trial  was  assigned 
to  one  of  three  conditions  with  respect  to  ECM:  Benign,  Design,  and  IOC.  In 
the  benign  condition,  enemy  forces  did  not  employ  ECM.  In  the  design 
condition,  the  ECM  conditions  faced  were  as  described  in  the  design  specifica¬ 
tions.  Finally,  the  third  ECM  condition  attempted  to  simulate  the  ECM 
environment  that  it  was  anticipated  the  Sgt  York  would  encounter  when  actually 
fielded;  that  is,  the  initial  operating  conditions  (IOC).  Thus,  the  three 
treatment  conditions  presented  increasingly  difficult  challenges  for  the 
weapon  system. 

For  the  two  conditions  in  which  ECM  was  encountered  (Design  and  IOC),  one 
of  three  different  versions  (designated  ECM  suites)  wa6  introduced  during 
each  trial.  That  is,  each  Design  trial  employed  Suite  1,  2,  or  3  and  each 
IOC  trial  likewise  was  assigned  to  one  of  three  suites.  To  implement  the 
Design  level  of  ECM,  A-lOs  and  A-7s  dropped  flares.  To  implement  the  IOC 
level  of  ECM,  A-lOs  dropped  flares  and  chaff  and  AH-64s  dropped  chaff  only. 

Table  32  presents  a  comparison  of  mean  times  for  the  11  measures  or  seg¬ 
ments  of  the  target  engagement  sequence  for  each  of  the  three  ECM  conditions 
represented  during  Sgt  York  FOE  I.  Differences  in  engagement  time  were 
significant,  and  differences  among  acqusition  times  were  close  to  the 
signficance  criterion.  For  acquisition  time,  the  mean  was  shortest  for  the 
Design  ECM  condition  and  longest  for  the  IOC  condition.  For  total  Engagement 
Time,  the  Design  ECM  condition  had  the  shortest  mean,  and  the  IOC  condition 
produced  the  longest  total  Engagement  Time. 
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Table  32 


Comparison  of  Target  Engagement  Task  Times  as  a  Function  of  ECM  Condition 


Target  Engagement  Task 

I  Benign  I 
ECM 
(s) 

Design  | 
ECM 
(s) 

njim 

mmm 

I  Significance 
Level 

Select 

6.8 

6.8 

8.9 

NS 

Pointer  On 

2.5 

2.2 

2.2 

NS 

Laser  On 

1.5 

1.1 

0.9 

NS 

Acquisition 

4.2 

3.0 

4.7 

.051 

Fire  Solution: 

Blinking 

5.2 

3.8 

3.6 

NS 

Fire  Solution: 

Solid 

2.6 

2.5 

2.6 

NS 

Fire  Decision: 

Blinking 

1.5 

1.8 

1.8 

NS 

Fire  Decision: 

Solid 

0.8 

0.9 

1.7 

NS 

Time  to  Fire 

16.2 

14.3 

17.6 

NS 

Fire  Duration 

8.2 

6.2 

8.8 

NS 

Engagement  Time 

24.4 

20.5 

26.4 

.022 

Time  of  Day  Comparisons 

The  15  trials  that  have  been  included  in  the  data  base  were  identified 
according  to  the  time  of  day  at  the  conclusion  of  each  trial.  Fourteen  of  the 
trials  involving  264  engagements  were  conducted  during  the  day.  Only  one 
trial,  which  involved  seven  engagements,  was  conducted  at  night.  The  night 
trial  was  No.  1040,  and  the  scenario  was  Road  March.  Recall  that  a  second 
Road  March  trial  was  included  among  the  15  trials  and  then  found  to  be  lacking 
in  the  movement  characteristic.  In  order  to  analyze  whether  there  were  any 
significant  differences  associated  with  time  of  day,  two  comparisons  were 
made.  The  first  comparison  was  between  the  14  day  trials  and  the  one  night 
trials.  The  second  comparison  in  effect  redefined  the  independent  variable. 
This  was  done  by  dividing  the  trials  into  three  time  clusters  which  represen¬ 
ted  (1)  early  morning,  (2)  midday,  and  (3)  afternoon  trial  times. 

Time  of  Day  Comparison  -  Day  Versus  Night.  The  night  trial  concluded  at 
2207  hrs.,  and  the  day  trials  ended  between  1049  hrs.  and  1800  hrs.  When 
these  trials  were  compared,  the  day  and  night  trials  were  found  to  be 
significantly  different  with  respect  to  number  of  targets  (p*.047)  and  fire 
solution  solid  cue  (p  -  .016). 

As  Table  33  indicates,  an  average  of  5.2  targets  were  observed  on  the 
plasma  display  at  the  initiation  of  the  engagement  sequence  during  day  condi¬ 
tions.  A  mean  of  2.5  targets  was  observed  on  the  plasma  display  for  the 
night  trial  at  the  initiation  of  the  engagement  sequence.  It  should  be  noted 
that  the  sample  size  for  number  of  engagements  during  the  night  condition  was 
extremely  small  in  comparison  to  daytime  engagements. 
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Table  33 

Time  of  Day  Comparisons  -  Day  Versus  Night  -  Number  of  Targets 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(s) 

Day  Trials 

264 

5.22 

3.69 

Night  Trials 

7 

2.48 

1.51 

Day  trials  also  differed  from  night  trials  (p=.016)  with  respect  to  time 
to  obtain  a  solid  fire  cue.  The  fire  solution  solid  time  is  associated  with 
system  processing  functions,  not  with  crew  actions  or  crew  decisions.  Radar 
Pointer  was  the  primary  system  mode  used  during  Trial  1040. 

During  the  night  condition,  it  took  the  fire  control  computer  4.5  seconds 
longer  than  during  the  day  condition  to  calculate  a  fire  solution  that 
produced  a  solid  cue  (see  Table  34).  This  weapon  system  processing  time  is 
not  indicative  of  crew  action  or  crew  decision  times. 


Table  34 


Time  of  Day  Comparisons  -  Day  Versus  Night  -  Fire  Solution  Solid 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(s) 

(s) 

Day  Trials 

264 

2.5 

3.2 

Night  Trials 

7 

7.0 

5.4 

The  fundamentals  of  operations  should  have  been  the  same  during  the  day 
versus  night  trials,  but  certain  techniques  would  vary  for  night  operations. 
The  Sgt  York  was  designed  to  acquire,  track,  and  engage  targets  at  night  as 
well  as  in  adverse  weather  conditions.  The  weapon  system's  radar  and  data 
processors  should  not  have  been  affected  by  night  visibility  conditions.  The 
weapon  system  was  designed  to  detect  and  classify  all  targets  at  extended 
ranges  during  night  conditions  in  radar  system  modes.  Optically  tracking  and 
engaging  the  target  requires  the  weapon  system  to  use  a  passive  image 
intensification  system.  The  optical  system  mode  is  dependent  upon  ambient 
light,  and  limited  visibility  could  degrade  the  capability  of  the  passive 
image  intensification  system  (optical  mode).  Such  a  degradation  of  the 
ability  to  engage  targets  during  night  operations  should  not  have  affected 
weapon  system  processing  (as  indicated  by  time  for  fire  solution  solid  cues) 
since  acquisition  system  modes  used  incorporated  radar.  Ability  to  engage 
targets  might  have  been  influenced  by  the  number  of  targets  available.  As 
noted  earlier  in  this  section,  day  and  night  trials  differed  in  the  number  of 
targets  displayed;  more  than  twice  as  many  targets  (5.2  versus  2.5)  appeared 
on  the  plasma  display  at  the  initiation  of  day  engagements  as  at  the  initia¬ 
tion  of  night  engagements. 
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One  other  difference  between  the  day  and  night  trials  is  the  range-at- 
first-appearance  for  targets  that  were  subsequently  fired  on.  For  day  trials, 
this  distance  varied  from  0  to  10  km;  but  for  the  night  trial,  the  range  was 
only  from  3.5  to  5.0  km. 

Time  of  Day  Comparisons  -  Early  Morning,  Midday,  and  Late  Day.  In  order 
to  compare  whether  the  time  of  day  impacted  on  differences  among  the  trials, 
the  trials  were  divided  and  clustered  into  three  categories.  The  first 
category  was  termed  the  early  morning  trials.  Early  trials  commenced  with 
Trial  1029  which  concluded  at  1049  hrs.  The  early  trials  consisted  of  Trials 
1027,  1023,  and  1029,  and  ranged  in  time  between  1049  hrs.  and  1140  hrs.  for 
their  completion.  The  second  category  was  termed  midday,  and  represented 
trials  conducted  in  the  early  afternoon.  Early  afternoon  trials  included 
1022,  1035,  and  1040,  and  ranged  in  time  at  completion  of  trial  between  1336 
hrs.  and  1530  hrs.  The  third  category  was  termed  late  afternoon.  Late 
afternoon  trials  consisted  of  Trials  1036,  1039,  and  1042,  and  ranged  in  time 
between  1732  hrs.  and  1800  hrs.  for  the  completion  of  the  trials.  When  time 
of  day  was  defined  as  early  morning,  early  afternoon,  and  late  afternoon,  an 
analysis  of  variance  showed  that  time  of  day  made  a  significant  difference 
(p-.040)  for  Time  to  Fire  and  Fire  Duration  (p*.032). 

Time  to  Fire  is  a  cumulative  measure  of  operator  and  machine  functions;  it 
represents  target  engagement  time  from  target  appearance  up  to  the  time  at 
which  a  crew  member  depressed  the  trigger.  This  could  be  in  response  to  a 
solid  or  a  blinking  firing  cue.  Even  though  a  solid  cue  indicated  a  higher 
probability  of  successful  engagement,  it  was  acceptable  for  a  crew  member  to 
fire  on  a  blinking  cue. 

For  the  late  day  trials,  the  mean  Time  to  Fire  was  5.5  seconds  longer  than 
for  the  midday  trials  (see  Table  35).  Consider  fatigue  as  a  factor  in  the 
increase  in  seconds  between  midday  and  late  day  engagements.  Since  the  midday 
trials'  mean  Time  to  Fire  was  1.9  seconds  less  time  than  the  mean  for  the 
early  trials,  it  becomes  difficult  to  invoke  fatigue  as  an  explanation 
because  of  this  inconsistent  effect.  As  noted  below.  Fire  Duration  was  also 
influenced  by  the  time  of  day,  but  the  total  engagement  time  was  not. 


Table  35 

Time  of  Day  Comparisons  -  Early  Morning,  Midday,  and  Late  Day  -  Time  to  Fire 


ENGAGEMENTS 

(n) 

MEAN 

(8) 

S.D. 

(s) 

Early  Morning  Trials 

50 

16.5 

9.9 

Midday  Trials 

65 

14.6 

10.8 

Late  Day  Trials 

45 

20.1 

12.6 

The  Fire  Duration  differences  shown  in  Table  36  were  significant  (p*.032) 
among  the  three  time-of-day  periods.  Fire  Duration  is  the  time  interval  from 
the  beginning  of  the  trigger  pull  until  the  end  of  the  engagement;  i.e.,  when 
either  the  operator  released  the  trigger  or  the  weapon  system  stopped  firing 
due  to  the  loss  of  a  radar  lock  on. 
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Table  36 

Time  of  Day  Comparisons  -  Early  Morning,  Midday,  and  Late  Day  -  Fire 
Duration 


ENGAGEMENTS 

(n) 

MEAN 

(8) 

S.D. 

(s) 

Early  Morning  Trials 

50 

9.5 

10.8 

Midday  Trials 

65 

10.2 

9.6 

Late  Day  Trials 

45 

5.8 

4.4 

The  mean  Fire  Duration  for  the  Late  day  trials  was  shortest  at  5.8 
seconds.  Clearly,  more  of  these  late  day  trials  conformed  to  the  recommended 
range  of  firing  durations  (3-7  seconds)  than  did  the  Early  morning  or  Midday 
trials. 

Crews  did  not  receive  feedback  as  to  whether  or  not  they  had  hit  the 
target  during  the  FOE  I  Force-on-Force  trials.  The  fire  units  were  credited 
for  target  kills  by  a  Monte  Carlo  method.  Nor  was  the  quantity  of  ammunition 
expended  or  remaining  revealed  to  the  crews.  This  lack  of  feedback  may  have 
encouraged  longer  Fire  Duration  times.  Perhaps  both  the  longer  overall  Time 
to  Fire  and  the  briefer  Fire  Durations  typical  of  the  late  day  trials  were 
the  result  of  fatigue. 

Target  Range  at  First  Appearance 

In  these  trials,  a  number  of  different  scenarios  and  tactics  were  used,  as 
has  been  discussed  in  the  preceding  pages.  It  is  consistent  with  this  that 
targets  would  appear  at  a  variety  of  ranges.  This  in  fact  was  the  case. 

Table  37  shows  the  variation  in  range  of  target  at  first  target  appearance. 

It  may  be  seen  that  targets  could  appear  at  immediate  proximity  to  the  fire 
unit  or  they  could  appear  as  far  as  10  km  away.  The  mean,  median,  and  mode 
values  tend  to  approximate  distances  of  3-4  km. 


Table  37 

Variations  in  Range  at  First  Target  Appearance 


Parameters 

Number 

Number  of  engagements 

271 

Minimum  range  (km) 

0 

Maximum  range  (km) 

10 

Mean  (km) 

3.6 

Median  (km) 

3 

Mode  (km) 

4 

Standard  Deviation  of  Mean  (km) 

2.4 
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However,  recalling  the  actual  frequency  distributions  of  range  at  first 
appearance  that  were  shown  in  Figure  23,  it  is  apparent  that  there  were  great 
variations  in  target  distance  on  first  appearance.  It  was,  for  example, 
highly  probable  that  the  target  would  appear  anywhere  within  4  km  distance 
from  the  fire  unit. 

It  is  to  be  expected  that  the  distance  of  the  target  should  have  had  some 
effect  on  performance  times.  It  seems  reasonable,  for  example,  that  the 
closer  the  target,  the  more  urgent  the  response  should  have  been  from  the  fire 
unit.  And  the  data  confirm  that  expectation.  Table  38  shows  that  the  range 
of  target  first  appearance  is  correlated  with  the  target  selection.  Time  to 
Fire,  and  Total  Engagement  Times.  While  the  correlations  are  not  large,  they 
are  significant.  It  could  be  predicted  that  the  closer  the  target  when  it 
first  appears,  the  faster  would  be  the  response  time  of  the  fire  unit.  And, 
for  those  functions  in  which  the  operator  was  involved,  it  did  appear  that 
faster  response  times  occured. 


Table  38 

Correlation  of  Target  Range  and  Subsystem  Tasks 


Task  Functions 

r  *  Range  of  Target 
First  Appearance 

Select 

.202  * 

Acquire 

.101 

Time  to  Fire 

.280  * 

Fire  Duration 

-0.036 

Engagement  Time 

.202  * 

*  Statistically  significant  p<.01 

For  example,  Figure  24  shows  Time  to  Fire  in  seconds  as  a  function  of  the 
range  of  the  targets  in  kilometers  up  to  distances  of  4.0  km.  It  may  be  seen 
that  there  is  an  increase  in  Time  to  Fire  as  the  range  increases.  It  appears 
that  there  might  be  a  linear  increasing  function. 

Perhaps  the  most  alarming  number  in  Figure  24  is  that  associated  with  a 
zero  target  range.  This  assumes  that  the  target  is  on  the  fire  unit  and 
presumably  most  threatening.  But,  the  mean  Time  to  Fire  was  almost  9  seconds. 
This  is  beyond  the  system  response  time  standard  (7  seconds)  and,  perhaps,  was 
beyond  a  comfortable  firing  response  time  for  a  hostile  target  so  close  to  the 
unit . 

In  system  models,  range  can  be  expressed  in  ways  other  than  distance.  For 
example,  range  could  be  translated  into  time  for  the  unit  to  deal  with  the 
threat.  If  this  is  done,  these  data  suggest  that  subsystem  and  task  response 
times  should  be  linked  with  that  distance-time  relationship.  But,  clearly. 


110 


ocher  considerations  may  be  paramount;  target  priorities  may  not  be  directly 
correlated  to  distance  from  the  fire  unit,  for  example. 

It  should  also  be  noted  that  the  only  measure  of  target  range  used  for 
this  analysis  was  range-at-first-appearance.  It  is  unlikely  that  crew 
members  always  noted  a  target  immediately;  other  measures,  such  as  range-at- 
first-crew-action,  might  show  a  stronger  relationship  to  Time  to  Fire.  Range 
at  first  sighting  (by  an  operator)  might  be  one  of  the  more  revealing 
measures,  but  to  have  such  a  measure  available  would  require  data  collection 
techniques  designed  to  indicate  when  the  operator  first  noted  the  target. 


Ill 


Range  First  Target  Appearance  (KM) 


Individual  and  Crew  Performance 


Crew  Action/Decision  Response  Times 

The  Sgt  York  target  engagement  sequence  (TES)  provides  an  opportunity  to 
look  at  some  specific  measures  of  operator  and  crew  performance.  Note  that 
the  representativeness  of  the  data  is  limited  in  that  it  was  generated  by  four 
Gunners,  and  five  Squad  Leaders.  Table  39  shows  the  crew  performance  mean 
times  both  overall  and  separately  by  crew  position.  Included  are  three 
measures  of  operator  action,  i.e..  Pointer  On  time,  Laser  On  time,  and  Fire 
Duration.  There  are  also  three  measures  of  operator  decision-making  time: 

(1)  the  time  it  takes  to  select  and  classify  a  target,  or  Select  time;  (2)  the 
time  it  takes  to  decide  to  fire  on  a  target  after  a  blinking  fire  cue  is 
presented,  or  Fire  Decision:  Blinking  time;  and  (3)  the  time  it  takes  to  fire 
on  a  target  after  a  solid  fire  cue  is  presented,  or  Fire  Decision  Solid  time. 
There  are  also  two  summary  measures  that  necessarily  include  components  both 
of  operator  performance,  and  of  hardware  or  software  functioning;  they  are 
Time  to  Fire  and  Total  Engagement  Duration. 

With  respect  to  the  initial  action  of  selecting  a  target,  the  squad 
leaders  as  a  group  showed  a  strong  preference  for  using  the  radar  pointer 
rather  than  the  laser.  Of  the  90  target  selections  performed  by  squad 
leaders,  all  but  one  selection  used  the  radar  pointer.  The  gunners  showed  no 
similar  preference.  Of  the  131  target  selections  carried  out  by  gunners,  68 
were  by  radar  pointer  and  63  by  lasing,  a  nearly  even  split.  For  the  three 
measures  of  crew  action  time,  pointer  on,  laser  on,  and  fire  duration,  the 
squad  leader's  times  are  on  the  average  shorter  than  the  gunner's,  but  these 
differences  are  not  statistically  significant.  The  difference  in  the  times 
used  by  the  squad  leader  and  the  gunner  in  making  the  target  selection 
decision  was  also  not  statistically  significant. 

The  only  difference  that  did  reach  statistical  significance  was  the 
length  of  time  it  took  to  respond  to  a  solid  firing  cue.  In  all  the  en¬ 
gagements,  it  was  the  gunner  who  fired,  but  if  he  had  also  taken  the  Initial 
action,  whether  pointing  or  lasing  the  target,  he  fired  more  quickly  than  if 
the  squad  leader  had  taken  the  initial  action.  On  gunner-initiated  engage¬ 
ments,  trigger  pull  occurred  a  mean  of  1.0  seconds  after  the  onset  of  the 
solid  firing  cue.  For  squad  leader-initiated  engagements,  it  took 
2.0  seconds  after  a  solid  fire  cue  appeared  for  a  firing  decision  to  be  made 
and  implemented.  Thus,  the  delay  between  the  onset  of  a  solid  firing  cue  and 
trigger  pull  was  about  one  second  greater  for  engagements  initiated  by  a  squad 
leader  than  for  engagements  initiated  by  a  gunner.  Evidently,  taking  the 
responsibility  for  both  operator  aspects  of  the  engagement  (selecting  the 
target  and  firing  on  it)  facilitated  rather  than  interfered  with  the  process, 
as  evidenced  by  a  savings  of  almost  a  full  second. 

The  reaction  to  a  blinking  fire  cue  did  not  show  the  same  effect.  As 
Table  39  indicates,  the  delay  in  responding  to  a  blinking  fire  cue  was 
greater  (rather  than  smaller)  on  gunner-initiated  engagements  than  it  was  on 
squad  leader-initiated  ones,  but  the  difference  was  not  statistically 
significant.  That  is,  who  initiated  the  engagement  had  no  real  (reliable) 
effect  on  time  taken  to  respond  to  a  blinking  fire  cue. 
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Table  39 


Crew  Action/Decision  Response  Times  Shown  as  a  Function  of  Position 


1  Over  | 
1  All  | 
(s) 

Squad 
|  Leader 
(8) 

I  Gunner  | 

1 

(8) 

Difference  Between 
SL  and  G 

Crew 

Pointer  On 

2.3 

2.2 

2.4 

NS 

Action 

Laser  On 

1.1 

0.3 

1.2 

NS 

Fire  Duration 

8.0 

7.5 

8.8 

NS 

Crew 

Select 

7.9 

8.4 

7.6 

NS 

Decision 

Fire  Decision: 

Blinking 

1.7 

1.5 

2.3 

NS 

Fire  Decision: 

Solid 

1.3 

2.0 

1.0 

.032 

Time  to  Fire 

16.5 

19.7 

17.3 

NS 

Engagement  Duration 

24.6 

27.2 

26.1 

NS 

Note  1.  The  gunner's  Fire  Decision  Solid  time  is  signficantly  shorter  than 
the  squad  leader's  time. 


In  further  comparing  times  for  gunner-initiated  engagements  with  those  for 
engagements  initiated  by  squad  leaders,  only  one  other  difference  was 
significant.  System  acquisition  time  was  significantly  shorter  (p**.035)  for 
gunner-initiated  engagements.  Table  40  shows  acquisition  mean  time  as  a 
function  of  which  crew  member  took  the  first  action.  There  is  no  evident 
reason  why  what  is  essentially  a  hardware/ software  function  should  be 
sensitive  to  which  operator  initiated  the  engagement. 


Table  40 


System  Acquisition  Time  as  a  Function  of  Engagement  Initiator 


Initiator 

ENGAGEMENTS 

MEAN 

HE2QHHM 

(n) 

(8) 

(s) 

Squad  Leader 

90 

4.3 

2.5 

Gunner 

131 

3.2 

4.3 

As  Table  39  shows,  on  all  of  the  crew  action/decision  measures  discussed 
so  far,  the  mean  for  all  engagements  falls  somewhere  between  the  squad 
leader's  and  the  gunner's  means.  For  the  summary  measures,  Time  to  Fire  and 
Engagement  Duration,  this  relation  does  not  hold  true.  The  mean  times  for  all 
engagements  are  briefer  than  the  times  for  either  squad  leader  or  the  gunner- 
initiated  engagements.  This  difference  occurred  because  the  overall  means 
include  system-initiated  engagements  (Radar  Auto  engagements  and  engagements 
in  which  the  automatic  system  preempts  the  target  selection  even  when  the  auto 
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mode  has  not  been  chosen  by  the  crew)  as  well  as  operator-initiated  engage¬ 
ments. 

As  discussed  earlier  in  this  chapter  (see  Levels  of  Automation/ System 
Modes) ,  such  system-initiated  engagements  have  shorter  Engagement  Durations 
than  those  involving  crew  actions.  The  effect  on  the  overall  mean  of  these 
shorter  engagements  is  reflected  in  the  times  entered  in  Table  39.  That  is, 
the  overall  mean  Time  to  Fire  and  overall  engagement  time  were  both  shorter 
than  comparable  measures  for  squad  leader  and  gunner-initiated  engagements. 

Mission  Oriented  Protective  Posture  (MOPP) 


As  part  of  the  FOE  1  test  plan,  levels  of  MOPP  clothing  requirements  were 
Included  to  reflect  the  conduct  of  combat  operations  in  a  threat  environment 
consisting  of  conventional  firing  and  air  attacks  along  with  nuclear  weapons 
used  with  chemical  and  biological  agents  (NBC  threat)  in  the  Airland  Battle 
2000.  The  selection  of  a  protective  posture  against  a  NBC  threat  is  dependent 
on  the  threat,  terrain,  mission,  weather,  etc.  The  mission  oriented  protec¬ 
tive  posture  (MOPP)  prescribes  how  and  when  protective  clothing  and  equipment 
should  be  worn. 

The  15  trials  under  analysis  for  this  study  included  13  of  the  trials 
where  crew  members  were  operating  at  MOPP  Level  1.  Two  of  the  trials  (1027 
and  1029)  were  conducted  under  MOPP  Level  4  conditions.  For  purposes  of 
analysis,  MOPP  Level  1  trials  were  compared  to  MOPP  Level  4  trials.  Clothing 
requirements  for  MOPP  Level  1  specify  that  overgarments  may  be  worn  open  or 
closed  according  to  the  temperature.  Overboots,  mask/hood,  and  gloves  are  to 
be  carried.  At  MOPP  Level  4,  overgarments,  overboots,  masks/  hoods,  and 
gloves  are  all  to  be  worn  in  closed  configuration.  Maximum  protection  is 
achieved  at  MOPP  Level  4.  (Levels  2  and  3  are  considered  intermediate  stages 
of  protection.)  Table  28  shows  that  MOPP  Level  4  applied  during  Trials  1027 
and  1029.  All  other  Force-on-Force  trials  were  conducted  with  crew  members 
starting  the  trial  wearing  MOPP  gear  around  their  lower  legs  with  overboots 
on.  The  tactical  operating  procedure  was  to  suit  up  partially  before  entering 
a  vehicle.  Crew  members  wore  the  MOPP  overgarments  around  their  legs  for  a 
low-order  MOPP.  During  Trials  1027  and  1029,  the  MOPP  overgarments  were 
pulled  up  and  masks/hoods  and  gloves  were  donned . 

When  MOPP  Level  1  was  compared  with  MOPP  Level  4,  there  were  no  sig¬ 
nificant  differences  among  any  of  the  dependent  variables.  The  11  DVs  are: 
Select,  Pointer  On,  Laser  On,  Acquire,  Fire  Solution  Blinking,  Fire  Solution 
Solid,  Fire  Decision  Blinking,  Fire  Decision  Solid,  Time  to  Fire,  Fire 
Duration,  and  Engagement  Duration. 

The  lack  of  significant  differences  may  have  been  influenced  by  the 
limited  duration  of  the  trial  periods,  and  by  the  primary  system  mode  used 
during  Trials  1027  and  1029.  FOE  I  had  originally  been  planned  for  trials 
lasting  between  16  to  20  hours.  Trials  were  required  to  be  shortened  into  30- 
mlnute  periods  instead  of  up  to  20  hours  because  of  limits  in  the  data 
recording  system.  Thirty-minute  trials  do  not  produce  the  fatigue  that  would 
occur  in  a  20-hour  trial  conducted  in  MOPP  Level  4.  FM  44-11  indicates  that 
"...extended  periods  in  high  MOPP  will  result  in  decreased  efficiency  due  to 
physiological  and  psychological  effects."  Actually,  the  crews  were  in  the 
fire  units  for  period  of  2  to  3  hours  for  each  scenario  even  though  the  trials 
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were  confined  to  approximately  30  minutes.  Extended  trials  such  as  those 
originally  planned  would  probably  have  led  to  significant  differences  between 
MOPP  Level  1  and  MOPP  Level  4. 

If  more  engagements  had  been  conducted  under  the  system  acquisition  mode 
had  of  optical/laser,  the  opportunity  for  MOPP  4  to  influence  performance 
would  have  been  greater.  In  this  mode,  the  only  way  to  provide  target  range 
and  speed  data  to  the  fire  control  computer  is  by  lasing.  The  optical/laser 
mode  required  a  greater  number  of  steps  to  follow  as  well  as  more  demands  on 
dexterity.  When  the  engagements  were  combined  for  Trials  1027  and  1029,  there 
were  36  engagements  in  total.  Of  the  combined  total,  75%  of  the  engagements 
took  place  during  system  mode  A,  19%  in  system  mode  9,  3%  in  system  mode  C, 
and  3%  in  system  mode  8.  System  modes  A  and  9  are  both  variations  on  radar 
pointer.  They  are  not  representative  of  an  optical/laser  with  no  radar 
condition. 

Due  to  the  constraints  of  the  FOE  1,  it  is  not  possible  to  evaluate  the 
potential  degradation  in  performance  that  would  have  been  expected  during 
extended  trial  durations.  A  trial  which  compares  MOPP  clothing  requirements 
over  30-minute  intervals  does  not  adequately  compare  fatigue  levels  for 
extended  field  operations  conducted  under  MOPP  1-4  conditions. 

Target  Workload 

During  target  engagements,  the  operators  frequently  encountered  multiple 
targets  on  the  display.  Table  41  shows  some  of  the  target  statistics  per 
trial.  It  can  be  seen  in  the  table  that  from  1  to  18  targets  were  displayed 
simultaneously.  The  values  of  the  mean,  median,  and  mode  as  well  as  the 
standard  deviation  of  the  mean  indicate  that  there  was  much  variability  in  the 
number  of  targets  the  operator  might  see  during  a  given  engagement  sequence. 


Table  41 

Target  Workload  Data 


Parameters 

Number 

Number  of  engagements 

271 

Minimum  targets  per  trial 

1 

Maximum  targets  per  trial 

18 

Mean 

5.1 

Median 

4 

Mode 

1 

Standard  Deviation  of  Mean 

3.67 

Another  way  of  showing  the  target  workload  is  illustrated  in  Figure  25 
which  gives  the  frequency  distribution  of  targets  per  trial  extracted  from  all 
15  trials  analyzed.  As  the  table  shows,  the  most  frequent  target  workload  was 
Indeed  one  target,  but  it  was  very  probable  that  more  than  one  target  would  be 
on  the  display  at  any  time.  For  future  modeling,  target  workload  should  be  a 
relevant  model  parameter,  and  it  is  to  be  expected  that  the  operator  will  be 
presented  with  more  than  one  target  at  a  time.  It  is  difficult  to  envision  a 
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display  system  and  operational  scenario  for  air  defense  that  would  not  require 
a  variable  target  workload  for  the  operator. 

It  may  also  be  reasonably  assumed  that  variable  target  workload  would 
differentially  affect  operator  performance.  The  data  here  fail  to  show  any 
statistically  significant  performance  result.  Table  42  lists  the  correlations 
among  target  workload  and  the  five  major  subsystem  tasks.  As  may  be  seen  in 
that  table,  the  relationship  is  very  limited  and  may  be  chance  since  none  of 
the  correlations  are  statistically  significant. 

Another  way  of  looking  at  potential  performance  effects  is  shown  in  Figure 
26  which  shows  ttime  to  ffire  in  seconds  as  a  function  of  target  workload  up 
to  10  simultaneous  targets.  Supporting  the  correlation  result  from  the  Table 
42  (Time  to  Fire,  r  -  -0.030),  the  figure  shows  there  is  very  little  apparent 
effect  on  Time  to  Fire  as  target  workload  increases. 


118 


I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I 

oo>oor'<otn-+nM»-oc>«}r>'<Okn'*nN 


(SpUOO^S)  SJJJ  OJ_  »UJ  !1 


120 


Table  42 

Correlation  of  Target  Workload  and  Subsystem  Tasks 


Task  Functions 

r  =  Target  Workload 

Select 

0.113 

Acquire 

0.047 

Time  to  Fire 

-0.030 

Fire  Duration 

-0.055 

Engagement  Time 

-0.056 

Note.  No  correlations  are  statistically  significant. 

Further  consideration  of  the  particular  circumstances  for  this  system 
suggests  that  one  must  be  careful  not  to  assume  a  target  "workload"  variable 
just  because  there  are  multiple  targets  displayed.  In  looking  at  the  display 
configurations,  from  the  actual  sequences,  the  multiple  targets  were  often 
distributed  all  over  the  display  and  did  not  particularly  clutter  the  actual 
active  target.  A  measure  of  target  volume  more  meaningful  with  respect  to 
workload  might  be  definable.  For  example,  in  addition  to  counting  all  the 
targets  on  the  display  at  a  given  moment  (i.e.,  all  those  from  0  to  10  km 
range),  it  might  be  revealing  to  count  only  those  targets  within  a  kilometer 
of  the  target  of  interest.  Such  a  measure  might  reveal  an  underlying  workload 
phenomenon  with  respect  to  crew  action  or  decision  times  that  was  obscured  in 
the  present  analysis  by  disregarding  distance  from  the  active  target. 

Further,  the  display  system  designated  target  priorities  which  in  effect 
directed  the  operator  to  the  high-priority  target.  The  system  often  said: 
"Look  at  and  deal  with  target  X  and  don't  worry  about  all  those  other 
targets."  If  that  is  the  case  for  the  operator,  he  is  not  dealing  with 
multiple  equally  probable  targets  and  he  is  certainly  not  required  to  deal 
with  all  of  them.  "Workload"  is  a  term  that  applies  to  the  response  of  the 
operator  and  not  necessarily  the  independent  variables;  it  may  well  have  been 
in  this  system  that,  despite  many  displayed  targets,  there  were  not  many 
"workload”  problems  for  the  gunner  or  squad  leader.  All  of  this  argues  for 
much  care  in  the  definition  of  "target  workload."  It  is  possible  to  have 
multiple  targets  and  no  apparent  workload  problem  -  at  least  so  the  present 
data  suggest. 

Although  these  results  show  little  evidence  of  a  workload  phenomenon, 
recall  that  the  15  trials  analyzed  were  selected  because  they  included  the 
presence  of  firing  at  a  target.  Heavier  target  workloads  would  delay  or 
prevent  operators  from  firing  -  and  the  14  trials  that  did  not  involve  firing 
at  targets  were  excluded  from  this  analysis.  The  sample  of  trials  analyzed 
has  a  very  probable  bias  against  showing  the  effects  of  heavier  target 
workloads . 


121 


Personnel  Quality 


Personnel  Characteristics  and  Individual  Performance 


As  part  of  the  MANPRINT  initiative,  those  assessing  operational  tests 
have  been  directed  to  look  more  closely  at  the  characteristics  of  test 
players  and  at  how  operator  performances  are  related  to  individual  charac¬ 
teristics.  In  the  Sgt  York  FOE  I  tests,  as  in  almost  all  such  operational 
tests,  the  number  of  individuals  participating  as  operators  was  very  small. 

It  is  not  appropriate  or  reasonable  to  use  elaborate  statistics  with  data  of 
this  type  to  relate  personal  variables  to  operational  outcomes  when  so  few 
individuals  are  involved.  The  Sgt  York  test  provided  quantitative  data  on  the 
performance  of  five  Squad  Leaders  and  four  Gunners •  It  would  be  as  inap¬ 
propriate  to  neglect  it  as  it  would  be  to  make  too  much  of  it. 

Personnel  Characteristics.  In  the  Force-on-Force  phase  of  the  tests, 
there  were  14  players:  5  squad  leaders,  4  gunners,  and  5  drivers.  Excluding 
the  drivers ,  there  were  9  crew  members  who  had  roles  as  weapon  system 
operators  and  whose  performance  was  individually  reflected  in  the  data  base. 
Those  nine  players  are  identified  in  Table  43  and  some  general  descriptive 
data  are  provided. 


Table  43 

Sgt  York  Player  Personnel:  Identification,  Description,  and  Experience 


Player | Player!  MOS  |Time  in | Position  I  Time  In | Height ! Weight  I  Age 
ID#  Rank  MOS  S=Sqd  Ldr  Service  (in)  (lb)  (yr/mo) 

(yr/mo)  G*Gunner  (yr/mo) 


0240 

E5 

16L20 

1/6 

G 

5/2 

69 

161 

25/6 

1746 

E6 

S 

70 

150 

3753 

E7 

16L3H 

1/7 

S 

10/10 

72 

195 

28/11 

4343 

E6 

16L 

0/6 

s 

11/2 

73 

196 

29/2 

5197 

E5 

16L 

0/7 

G 

9/6 

69 

130 

27/7 

6111 

E5 

16L 

6/10 

G 

6/10 

73 

210 

24/9 

6640 

E6 

16L30 

0/7 

S 

9/11 

69 

169 

27/1 

9089 

E7 

16L4H 

1/6 

S 

14/3 

69 

130 

33/8 

9843 

E6 

16L30 

0/7 

G 

10/0 

74 

215 

28/6 

Table  43  identifies  each  crewman  by  player  identification  number  in 
Column  1  and  by  military  rank  in  Column  2.  Columns  3  and  4  represent  Military 
Occupational  Specialty  (MOS)  and  time  in  MOS  by  year  and  month.  MOS  16L  was 
identified  as  the  Sgt  York  Air  Defense  Gun  System  Crew  Member. 

Prerequisite  experience  and  background  were  required  for  the  test  participant 
16L  MOS  personnel.  Specifically,  16L10-0SUT  prerequisite  was  Active  Army, 
Grade  E4  and  below,  with  Operator/Foodhandler  (OF)  score  95  or  above,  and 
Electrical  (EL)  score  90  or  above;  and  16L/20/30/40-T  prerequisite  was  Active 
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Army,  Grade  E5  and  above,  related  Air  Defense  Artillery  (ADA)  MOS,  OF  score  95 
or  above,  and  EL  score  90  or  above. 


Column  5  of  the  table  specifies  the  crew  member's  position  within  the 
fire  unit.  Column  6  identifies  the  total  amount  of  time  each  crew  member  was 
in  the  service  by  year  and  by  month  at  the  time  of  FOE  I.  Columns  7  and  8 
give  the  height  and  weight  of  each  crew  member.  Performance  and  selection 
implications  are  associated  with  height  and  weight  measures  since  personnel 
above  the  60th  percentile  did  not  have  adequate  space  within  the  M48  tank 
chassis  (Korean  War  era  tank  chassis).  Column  9  describes  each  crew  member 
according  to  age  by  year  and  by  month. 

Individual  Performance.  Because  of  the  instrumentation  available  on  the 
Sgt  York,  Select  and  Classify  time,  pointer  on  time,  and  laser  on  time  were 
measurable  and  also  could  be  attributed  to  a  specific  crew  member.  A  com¬ 
parison  of  individual  crew  member  performance  was  conducted  for  gunners  and 
squad  leaders  regarding  Select  and  Classify  target  times  and  pointer  on  times. 
Laser  on  times  were  not  available  for  all  crew  members,  so  no  statistical 
comparison  was  made  on  that  measure. 

Select  Target  time.  Select  and  Classify  target  was  defined  as  the  amount 
of  time  it  took  a  crew  member  to  detect  and  begin  to  respond  to  a  target  from 
the  time  the  target  appeared  on  the  display  to  the  time  the  pointer  was 
depressed.  This  period  of  time  was  considered  crew  decision  making.  An 
analysis  of  variance  was  performed  comparing  Select  and  Classify  target  times 
for  individual  crew  members.  Findings  indicated  that  there  was  a  significant 
difference  (p=.007)  among  individual  crew  members.  Individual  select  scores 
are  presented  in  Table  44. 


Table  44 

Select  and  Classify  Target  Scores  for  Individual  Crew  Members 


ENGAGEMENTS 

(n) 

Mean 

(s) 

Gunner ,  #0240 

36 

5.0 

Gunner,  #5197 

33 

7.3 

Gunner,  #6111 

31 

13.7 

Gunner,  #9843 

35 

4.7 

Squad  Leader,  #1746 

26 

12.0 

Squad  Leader,  #3753 

15 

9.6 

Squad  Leader,  #4343 

19 

5.6 

Squad  Leader,  #6640 

12 

7.6 

Squad  Leader,  #9089 

15 

5.6 

Two  gunners  (#9843  &  #0240)  were  faster  at  selecting  and  classifying 
targets  than  were  the  other  crew  members.  Their  select  and  classify  times 
averaged  4.7  seconds  and  5.0  seconds,  respectively. 

Pointer  on  time.  Another  comparison  of  individual  crew  member  performance 
was  conducted  for  gunners  and  squad  leaders  and  the  time  in  seconds  it  took 
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them  to  get  a  target  pointed.  Pointing  the  target  wa6  considered  a  crew 
action  time.  The  activity  consisted  of  the  crew  member  tracking  the  target 
while  holding  the  pointer  switch  depressed  during  the  radar  pointer  mode.  The 
crew  member  designated  the  target  on  the  display  with  a  cursor.  When  the 
pointer  on  symbol  touched  the  target  symbol,  the  pointer  on  button  was 
released  by  the  computer.  Pointer  on  time  represents  the  amount  of  time  it 
took  a  crew  member  to  get  a  target  pointed.  Individual  pointer  on  mean  times 
are  shown  in  Table  45.  Analysis  of  variance  comparing  gunners  and  squad 
leaders  by  pointer  on  time  indicated  that  the  differences  were  significant. 
(P-.003). 


Table  45 

Pointer  On  Scores  for  Individual  Crew  Members 


ENGAGEMENTS 

(n) 

Mean 

(s) 

Gunner,  #0240 

15 

1.8 

Gunner,  #5197 

17 

3.2 

Gunner,  #6111 

14 

2.3 

Gunner,  #9843 

23 

2.4 

Squad  Leader,  #1746 

26 

2.3 

Squad  Leader,  #3753 

14 

1.6 

Squad  Leader,  #4343 

19 

1.3 

Squad  Leader,  #6640 

12 

3.8 

Squad  Leader,  #9089 

15 

1.6 

In  the  Pointer  On  crew  activity,  four  of  the  crewmen  had  mean  pointer  on 
times  that  were  under  2  seconds.  The  highest  mean  of  3.8  seconds  was  almost 
three  times  the  shortest  mean  of  1.3  seconds. 

Having  individual  performance  measures  and  discovering  that  there  are 
significant  differences  in  performance  scores  are  only  the  beginning.  The 
next  step  is  to  search  for  individual  characteristics,  ratings,  or  experience 
that  vary  with  the  performance  measures,  and  thereby  enable  prediction  of  the 
performance  from  the  characteristic,  etc.  The  pre-established  selection 
criteria  for  attendance  in  the  MOS  16L  transition  course,  discussed  at  the 
beginning  of  this  section,  included  (1)  Active  Army,  (2)  Grade  E5  and  above, 
(3)  related  ADA  MOS,  (4)  OF  score  of  at  least  95,  and  (5)  EL  score  of  at  least 
90.  The  first  two  criteria  were  not  variables.  The  third  criterion  was 
translated  into  time  in  MOS,  but  there  was  no  apparent  relationship  between 
the  time  crew  members  had  been  in  their  MOS  and  mean  Select  or  Pointer  times. 

The  other  individual  descriptors  included  in  Table  43  were  inspected,  but 
these  descriptors  do  not  seem  to  be  related  to  performance  patterns.  Perhaps 
if  the  sample  had  been  larger,  there  might  have  been  some  descriptors  that 
would  have  proven  useful  in  selecting  crew  members  with  potential  for  better 
performance  on  target  engagement  tasks. 

Additional  data  were  available  for  most  of  the  Sgt  York  player  personnel, 
Including  level  of  education,  OF  and  EL  scores  on  the  Armed  Services  Vocation¬ 
al  Aptitude  Battery  (ASVAB),  Armed  Forces  Qualification  Test  (AFQT)  scores, 


mental  category,  and  GT  scores.  Although,  as  noted  earlier,  no  elaborate 
statistical  analyses  are  appropriate  with  such  limited  data.  Table  46  was 
constructed  to  facilitate  inspection  of  the  relationship  between  the  various 
descriptive  scores  listed  above  and  individual  performance  during  the 
operational  tests.  Thus,  Table  46  Includes  select  time,  pointer  on  time,  and 
laser  on  time. 


Table  46 


Sgt  York  Player  Personnel:  Characteristics  and  Performance  Scores 


Player  | 
ID# 

|  Level  of  | 
Education 

I  ASVAB  |AFQT 
Score 

OF  |  EL 

|  Mental  | 
Category 

GT 

Score 

| Select/ 
Classify 

X 

(8) 

IPointer 

On 

X 

(8) 

|  Laser 
On 

X 

(8) 

0240 

GED 

98 

103 

35 

IIIB 

80 

5.0 

1.8 

1.5 

1746 

— 

— 

— 

65 

II 

109 

12.0 

2.3 

— 

3753 

C2 

112 

113 

65 

II 

118 

9.6 

1.6 

0.3 

4343 

Cl 

105 

109 

— 

IIIA 

99 

5.6 

1.3 

— 

5197 

HSD 

95 

109 

59 

II IA 

96 

7.3 

3.2 

1.2 

6111 

HSD 

105 

93 

19 

IV 

87 

13.7 

2.3 

0.9 

6640 

HSD 

114 

76 

25 

IV 

84 

7.6 

3.8 

— 

9089 

HSD 

118 

106 

23 

IV 

108 

5.6 

1.6 

— 

9843 

GED 

119 

120 

65 

II 

110 

4.7 

2.4 

0.9 

Note.  The  abbreviations  used  in  this  table  have  the  following  meanings: 


ASVAB  -  Armed  Services  Vocational  Aptitude  Battery 
OF  ■»  Operator/ Foodhandler 

EL  *  Electronics 

AFQT  *  Armed  Forces  Qualification  Test 

GT  -  General  Test 

GED  **  General  Education  Diploma 

C2  *  College,  two  years 

Cl  -  College,  one  year 

HSD  ■  High  School  Diploma 

Level  of  education.  For  all  but  one  crew  member,  level  of  education  is 

Indicated  in  Table  46.  Level  of  education  was  graphed  against  mean  Select 

time  for  eight  crew  members  (Figure  27).  The  figure  indicates  that  the  mean 
select  time  did  not  go  down  as  education  went  up.  Indeed,  the  two  operators 
who  had  a  General  Education  Diploma  (GED),  which  was  considered  the  lowest 
level  of  education  represented  in  the  sample  of  eight,  had  the  shortest 
select/classify  mean  times.  Short  select  times  were  achieved  by  operators  in 
each  category  (GED,  High  School  Diploma  (HSD),  and  College  (C)). 


Level  of  education  was  also  plotted  against  mean  Point  time.  As  Figure  28 
shows,  fast  Point  times  were  found  among  individuals  representing  all  levels 
of  education  (GED,  HSD,  and  C).  As  a  group,  high  school  graduates  seemed  to 
be  slowest  at  pointing,  but  more  data  would  be  needed  to  substantiate  the 
curvilinear  relationship  suggested.  Amount  of  formal  education  does  not 
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indicate  how  well  a  crew  member  will  perform  the  tasks  that  contribute  to 
tracking  a  target. 

ASVAB  scores.  Crew  members  scores  on  the  OF  and  EL  scales  of  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB)  are  included  in  Table  46.  To 
look  at  the  effect  of  these  remaining  two  selection  criteria  (OF  score  and  EL 
score),  mean  Select  time  was  plotted  against  individual  OF  scores  (Figure  29) 
and  against  individual  EL  scores  (Figure  30).  Note  that  one  squad  leader 
(Player  #6640)  was  below  the  EL  cutoff  of  90  (discussed  earlier).  Neither 
ASVAB  OF  nor  ASVAB  EL  scores  demonstrated  a  consistent  relationship  between 
higher  ASVAB  scores  and  lower  Select  times. 

Mean  Pointer  On  times  were  similarly  plotted  against  ASVAB  OF  scores 
(Figure  31)  and  ASVAB  EL  scores  (Figure  32).  Individual  differences  in  OF 
and  EL  scores  did  not  appear  to  account  for  differences  in  point  time.  When 
the  four  crew  members  with  the  best  mean  times  for  pointing  a  target  were 
compared  on  ASVAB  OF  scores,  the  scores  ranged  between  98  and  118.  If  all 
crew  members  were  Included  in  the  range  of  scores,  the  scores  ranged  between 
95  and  119.  That  is,  the  OF  scores  for  the  best  performers  covered  almost  the 
full  range. 

The  ASVAB  Operator/Foodhandler  (OF)  consists  of  two  subtests  from  the 
Armed  Services  Vocational  Aptitude  Battery.  These  subtests  were  used  to 
select  Vulcan  crewmen.  Average  ASVAB  OF  scores  for  Vulcan  crewmen  (N*1281) 
was  100.  The  average  for  the  Sgt  York  sample  of  eight  crew  members  for  whom 
scores  were  available  was  108.  As  noted  earlier,  prerequisite  scores  had  been 
established  for  future  16L  MOS  personnel  which  included  an  Operator/Food¬ 
handler  (OF)  score  of  95  or  above. 

EL  scores  for  the  four  crew  members  with  the  best  mean  times  for  pointing 
a  target  ranged  between  103  and  113.  When  all  the  EL  scores  were  taken  into 
account  for  the  crew  members,  the  scores  ranged  between  76  and  120. 

From  this  small  sample,  it  does  not  appear  that  ASVAB  OF  or  EL  scores  are 
useful  in  predicting  performance.  For  a  discussion  of  the  ASVAB  scores  and 
observations  related  to  training,  see  Seven  (1987).  Whether  the  ASVAB  cutoff 
scores  were  too  high  or  did  not  indicate  enough  about  individual  ability  was 
not  resolved. 
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Mean  Point  Time  (Seconds)  ^  Mean  Select  Time  (Seconds) 


Level  of  education 


Figure  28.  Mean  point  time  in  seconds  by  level  of  education. 
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ASVAB  Operotor/Foodhandler  (OF)  Scores 

Figure  31.  Individual  OF  scores  plotted  against  mean  point  time. 


ASVAB  Electronics  (EL)  Scores 

Figure  32.  Individual  EL  scores  plotted  against  mean  point  time. 
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AFQT  scores.  No  prerequisites  had  been  established  with  respect  to  AFQT 
score.  When  individual  crew  member  AFQT  scores  were  plotted  by  mean  Select 
time,  the  distribution  Indicated  that  higher  AFQT  scores  may  not  necessarily 
be  associated  with  lower  performance  time  as  indicated  by  time  to  classify  and 
select  a  target.  There  was  a  difference  of  30  points  between  crew  members 
#0240  and  #9843  on  AFQT  score,  even  though  their  mean  Select  and  Classify 
target  times  were  similar.  As  Figure  33  illustrates,  there  does  not  appear  to 
be  a  simple  relationship  between  AFQT  score  and  performance  as  indicated  by 
Select  time. 

AFQT  scores  of  Sgt  York  operators  were  plotted  against  crew  member  mean 
Point  time.  As  Figure  34  shows,  the  resulting  graph  indicated  that  there  was 
no  relationship  between  AFQT  score  and  the  time  it  took  a  crew  member  to  point 
a  target.  The  findings  suggest  that  AFQT  scores  may  not  be  useful  in 
selecting  crew  members  for  activities  that  focus  on  pointing  a  target. 

Note  (see  Table  46)  that  the  lowest  AFQT  score  (19)  for  this  sample  was 
obtained  by  a  Sgt  York  gunner.  This  individual  also  had  the  highest  in¬ 
dividual  training  score.  This  observation  supports  the  conclusion  that 
soldiers  with  low  AFQT  scores,  or  some  subset  of  them,  are  capable  of 
satisfactorily  completing  individual  and  collective  training,  as  well  as 
being  capable  of  operating  a  complex  weapons  system  during  a  field  test 
situation. 

AFQT  scores  were  also  related  to  crew  member  mean  performance  time  in 
seconds  for  pointing  a  target.  The  range  in  AFQT  scores  for  crew  members 
with  a  high  level  of  performance  in  mean  time  to  point  a  target  ranged 
between  scores  of  23  and  65.  All  of  the  crew  members  used  in  the  sample  in 
this  study  had  AFQT  scores  ranging  from  19  through  65.  An  average  AFQT  score 
had  been  obtained  on  the  population  of  16R  (Vulcan)  crewmen.  The  Vulcan 
average  AFQT  score  was  45.4  (N-1281).  The  average  AFQT  score  for  the  eight 
Sgt  York  crew  members  for  whom  scores  were  available  was  44.5,  suggesting  that 
on  this  measure,  at  test,  the  Sgt  York  operators  were  fairly  representative. 

Mental  Category  classification.  A  Mental  Category  histogram  was  con¬ 
structed  to  explore  whether  a  relationship  existed  between  Mental  Category 
classification  and  Individual  crew  member  mean  select  and  classify  target 
time.  As  Figure  35  shows,  individual  performance  times  were  quite  similar, 
and  rapid  selection  was  achieved  by  individuals  in  Mental  Categories  II,  IIIA, 
IIIB,  and  IV.  Knowledge  of  Mental  Category  does  not  appear  to  be  important  in 
the  selection  of  personnel  for  this  task.  Results  based  on  the  small  sample 
of  nine  crew  members  would  allow  for  the  participation  of  crew  members  with  a 
Mental  Category  IV  classification.  Such  an  outcome  could  broaden  the  pool  for 
candidate  crew  members. 

Mental  Category  classification  was  also  plotted  against  Pointer  On  time 
for  each  individual  crew  member.  The  four  crew  members  who  had  pointer  on 
times  under  2  seconds  had  Mental  Category  classifications  of  IIIA,  II,  IV,  and 
IIIB,  respectively.  When  mean  pointer  on  time  in  seconds  was  plotted  by 
Mental  Category  for  the  crew  members  in  this  study,  as  shown  in  Figure  36,  it 
became  apparent  that  the  slower  performers  on  the  crew  activity  of  pointing 
the  target  were  found  in  Mental  Categories  IIIA  and  IV.  However,  superior 
performance  for  mean  time  in  seconds  (l.e.,  quick  performance)  regarding  the 
pointer  on  task  was  found  in  Mental  Categories  II,  IIIA,  IIIB,  and  IV.  The 
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Figure  35.  Mean  select  tine  in  seconds  by  Mental  Category. 
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Figure  36.  Mean  point  tine  in  seconds  by  Meantal  Category. 
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histogram  indicates  that  Mental  Category  was  not  useful  for  selecting 
Individuals  who  would  exhibit  greater  skill  in  pointing  a  target.  Perhaps  if 
more  subjects  had  been  included.  Mental  Category  might  have  proven  valuable  in 
selecting  crew  members.  The  current  findings  suggest  that  it  may  be  ap¬ 
propriate  to  select  future  crew  members  for  complex  weapon  systems  such  as 
Sgt  York  from  Mental  Categories  II,  III,  and  IV. 

General  Test  scores.  As  with  AFQT  scores,  crew  members  #0240  and  #9843 
represent  the  bottom  and  the  top  of  the  distribution.  Also,  as  with  AFQT 
scores.  General  Test  (GT)  scores  showed  a  difference  of  30  points  between  crew 
members  #0240  and  #9843.  A  graph  was  constructed  to  display  the  spread  of  GT 
scores  for  individual  crew  members  by  the  mean  time  to  select  and  classify  a 
target.  As  Figure  37  shows,  a  high  GT  score  was  not  necessarily  associated 
with  quick  select/classify  performance  time. 

General  Test  scores  also  were  assessed  for  their  relationship  to  the  crew 
activity  of  pointing  a  target.  Crew  members  #0240,  #3753,  #4343,  and  #9089 
had  GT  scores  ranging  from  80  to  118.  These  crew  members  were  all  able  to 
point  a  target  in  less  than  2  seconds.  When  all  crewmen  were  identified  by  GT 
score,  scores  ranged  between  80  through  118.  Even  though  both  ranges  are 
identical,  a  graph  of  GT  scores  by  mean  point  time  in  seconds  indicated  that 
there  may  be  a  slight  trend  (and  a  bimodal  distribution)  toward  higher  GT 
scores  being  associated  with  a  higher  level  of  performance  as  reflected  in 
mean  time  in  seconds  to  point  a  target  (see  Figure  38).  Further  research 
would  be  warranted  in  developing  selection  criteria  that  include  GT  scores  for 
crew  activities  based  on  mean  performance  times.  The  limited  sample  size  does 
not  allow  for  generalizing  this  finding  with  any  confidence. 

Summary.  The  evidence  supports  the  contention  that  performance  time  in 
seconds  to  select  and  classify  a  target  is  not  related  to  available  crew 
descriptors  or  selection  scores.  Possibly,  other  selection  scores  could  be 
developed  which  would  have  an  element  of  predictability.  At  the  present 
time,  these  descriptors  and  selection  scores  will  not  be  of  great  use 
regarding  select/classify  tasks.  The  most  important  finding  for  performance 
time  on  the  selection  and  classification  of  the  target  was  associated  with 
Mental  Category  classification,  but  it  dealt  not  with  a  difference  but  the 
lack  of  a  difference.  Since  Mental  Categories  II,  IIIA,  IIIB,  and  IV  had 
similar  performance  times  on  select/classify  tasks,  further  MANPRINT  studies 
will  be  needed  to  assess  the  potential  for  increasing  the  use  of  Mental 
Category  IV  Individuals.  Data  presented  in  this  section  are  not  sufficient  to 
assess  personnel  selection  criteria,  but  they  do  indicate  that  personnel 
selection  criteria  should  be  reevaluated. 


Crew  Mix/Mental  Category 

During  the  Sgt  York  FOE  I  Force-on-Force  trials,  five  different  squad 
leaders  and  four  different  gunners  served  as  Sgt  York  crew  members.  An 
additional  gunner  was  selected  and  trained,  but  for  some  unrecorded  reason 
failed  to  take  part  in  the  test  trials.  Since  there  were  five  fire  units 
taking  part  in  the  tests,  the  four  gunners  were  moved  about  among  the  fire 
units,  creating  a  variety  of  squad  leader/gunner  mixes. 

As  noted  earlier  in  this  chapter  (see  Fire  Unit  Comparisons),  nine  unique 
crews,  or  squad  leader/gunner  mixes,  were  created.  The  performance  of  these 
crews  was  reviewed  and  compared.  The  one  significant  difference  found  among 
crews  was  on  mean  Time  to  Fire.  Time  to  Fire  means  and  standard  deviations 
for  each  of  the  crews  are  presented  in  Table  47;  the  nine  unique  crews  are 
designated  A  through  I  and  the  number  of  trials  in  which  they  took  part,  the 
number  of  complete  engagements  which  they  carried  out,  and  the  percentage  of 
the  engagements  initiated  by  the  crew  are  indicated  in  the  same  table. 


Table  47 

Mean  Time  to  Fire  for  Nine  Unique  Crews  (Squad  Leader/Gunner  Mixes) 


Crew 

Designation 

I  Time 

Mean  (s) 

to  Fire 

1  S.D.  (s) 

1  Number  of  | 
1  Trials 

Number  of  | 
Engagements 

%  Crew- 
Initiated 

A 

19.7 

14.4 

13 

37 

86% 

B 

12.2 

9.2 

9 

56 

71% 

C 

17.1 

12.5 

12 

65 

83% 

D 

18.5 

12.7 

12 

41 

90% 

E 

17.2 

8.2 

3 

15 

80% 

F 

14.6 

14.1 

2 

4 

50% 

G 

20.5 

14.8 

6 

18 

94% 

H 

15.8 

8.6 

2 

23 

70% 

I 

12.1 

5.5 

1 

12 

92% 

Note.  Crew 

Squad  Leader  ID 

Gunner  ID 

A 

9089 

6111 

B 

4343 

0240 

C 

1746 

9843 

D 

6640 

5197 

E 

3753 

5197 

F 

3753 

9843 

G 

3753 

0240 

H 

3753 

6111 

I 

4343 

9843 

Since  crew-initiated  engagements  took  longer  on  the  average,  the  shortest 
mean  times  to  fire  might  be  expected  to  have  been  associated  with  the  lowest 
proportion  of  crew  initiations.  Crew  I  had  the  shortest  mean  Time  to  Fire 
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(12.10  seconds);  Crew  G  had  the  longest  (20.52  seconds).  Both  had  approximat¬ 
ely  the  same  proportion  of  crew-initiated  engagements  (92%  and  94%,  respec¬ 
tively),  above  the  average  for  the  271  total  engagements  (82%).  Whether  or 
not  the  quicker  automatic  engagements  artificially  lowered  the  average  mean 
Time  to  Fire  for  some  crews,  it  is  unlikely  that  they  accounted  for  the 
difference  between  the  performance  of  Crews  G  and  I,  the  slowest  and  the 
fastest  crews. 

The  assignment  of  individuals  to  fire  units  and  hence  to  crews  was 
detailed  earlier  (see  Fire  Units)  and  is  further  discussed  below.  (The  in¬ 
dividual  crew  members  were  characterized  and  their  differences  discussed  in 
the  preceding  section  of  this  report.)  It  should  be  noted  that  the  squad 
leader  was  the  same  for  the  two  crews  with  the  fastest  Time  to  Fire  scores. 
That  squad  leader  belonged  to  Mental  Category  III.  In  Crew  I,  he  was  paired 
with  a  Category  II  gunner;  in  Crew  B  the  gunner  belonged  to  Category  IIIB. 

Crew  G,  the  one  with  the  slowest  mean  Time  to  Fire  score,  had  the  same 
Category  IIIB  gunner,  but  in  Crew  G  he  was  paired  with  a  Category  II  squad 
leader.  An  attempt  to  assess  the  overall  impact  of  the  Mental  Category  of 
crew  members  on  system  performance  was  made  earlier  in  this  section.  Mental 
Category  (a  nomenclature  changed  to  "Test  Score  Category"  as  of  31  October 
1987)  is  sometimes  presumed  to  be  related  to  an  individual's  ability  to 
function  as  part  of  a  complex,  highly  sophisticated  man-machine  system. 

Hence,  the  Mental  Category  was  one  variable  considered  in  viewing  performance 
scores. 

Table  48  shows  how  crew  members  were  assigned  to  fire  units  trial  by 
trial.  As  the  table  indicates,  with  one  exception,  each  squad  leader  stayed 
with  a  given  fire  unit  throughout  the  trials  analyzed.  The  one  exception  was 
on  Trial  1048;  the  crew  that  had  operated  Fire  Unit  2  on  eleven  earlier 
trials,  operated  Fire  Unit  4  on  the  final  trial  analyzed.  Three  of  the  squad 
leaders  were  paired  with  only  one  gunner  each.  Another  squad  leader  worked 
with  a  second  gunner  on  one  trial  only.  The  fifth  squad  leader  worked  with 
all  four  gunners,  each  for  at  least  two  trials. 

That  circumstance  provided  an  unusual  opportunity  to  compare  the  perfor¬ 
mance  of  individuals  belonging  to  different  Mental  Categories.  There  were 
gunners  belonging  to  Categories  II,  IIIA,  IIIB,  and  IV,  and  all  four  had 
worked  with  a  single  squad  leader  and,  although  in  different  trials,  on  the 
same  fire  unit.  The  results  of  this  comparison  are  presented  in  Table  49.  As 
the  table  indicates,  none  of  the  differences  were  significant.  In  other 
words,  in  this  context  and  with  these  individuals.  Mental  Category  was  not  a 
factor  in  speed  of  performance;  Categories  II,  III,  and  IV  gunners  performed 
equivalently. 

The  data  from  Sgt  York  FOE  I  provide  an  additional  opportunity  to  look  at 
the  effect  of  Mental  Category  on  performance.  Categories  II,  III,  and  IV  were 
represented  by  squad  leaders  as  well  as  by  gunners.  Furthermore,  there  were 
crews  in  which  both  members  were  Category  II,  another  in  which  both  members 
were  Category  III,  and  yet  another  in  which  both  members  were  Category  IV. 
Isolating  the  performance  of  these  crews  made  possible  the  comparison 
presented  in  Table  50.  Here  there  were  significant  differences  in  performance 
times.  As  the  table  shows,  select  time,  pointer  and  laser  on  times,  and  Time 
to  Fire  all  showed  significant  differences. 
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Table  48 


Assignment  of  Squad  Leaders  and  Gunners  to  Fire  Units  by  Trial 


Trial  1 

Fire  Unit  1 
SL  9089 

|  Fire  Unit  2 
SL  6640 

I  Fire  Unit  3 
SL  3753 

|  Fire  Unit  4 
SL  1746 

|  Fire  Unit  5 
SL  4343 

1020 

G  6111 

G  5197 

G  9843 

G  0240 

1021 

G  5197 

G  6111 

G  9843 

G  0240 

1022 

G  5197 

G  6111 

G  9843 

G  0240 

1023 

G  6111 

G  5197 

G  0240 

G  9843 

1026 

G  6111 

G  5197 

G  9843 

G  0240 

1027 

G  6111 

G  5197 

G  9843 

G  0240 

1029 

G  6111 

G  5197 

G  9843 

G  0240 

1035 

G  6111 

G  5197 

G  0240 

G  9843 

1036 

G  6111 

G  5197 

G  0240 

G  9843 

1039 

G  6111 

G  5197 

G  0240 

G  9843 

1040 

G  6111 

G  5197 

G  0240 

G  9843 

1041 

G  6111 

G  5197 

G  9843 

G  0240 

1042 

G  6111 

G  5197 

G  9843 

G  0240 

1046 

G  6111 

G  0240 

G  9843 

1048 

G  6111 

G  9843 

SL  6640 

G  0240 

G  5197 

Table  49 


Mean  Times  for  Target  Engagement  Sequence  Tasks  as  a  Function  of  Gunners  from 
Different  Mental  Categories  (II,  IIIA,  IIIB,IV),  All  Paired  with  the  Same 
Squad  Leader 


Category  of  Gunner 

1  II  1 

(s) 

!  IIIA  | 
(s) 

IIIB  1 
(s) 

IV 

(s) 

I  Significance 
Level 

Select 

14.8 

6.5 

8.2 

8.5 

NS 

Pointer  On 

2.2 

2.2 

1.8 

2.3 

NS 

Laser  On 

— 

0.9 

0.8 

1.4 

— 

Acquisition 

4.2 

4.2 

5.0 

5.6 

NS 

Fire  Solution:  B 

3.3 

3.2 

3.5 

1.3 

— 

Fire  Solution:  S 

0.9 

4.5 

3.6 

1.9 

NS 

Fire  Decision:  B 

0.7 

1.5 

2.5 

0.7 

— 

Fire  Decision:  S 

0.4 

2.0 

2.7 

1.1 

NS 

Time  to  Fire 

14.6 

17.2 

20.5 

15.8 

NS 

Fire  Duration 

7.0 

5.2 

9.0 

6.2 

NS 

Engagement  Time 

21.7 

22.4 

29.5 

22.0 

NS 

No.  of  Engagements 

4 

15 

18 

23 

137 


Table  50 


Mean  Times  for  Target  Engagement  Tasks  Presented  by  Mental  Category  Level  of 
Crew  Members 


Task 

|  Category  | 

II  Crew 
(8) 

Category  | 

III  Crew 
(8) 

Category  | 

IV  Crew 
(8) 

Significance 

Level 

Select 

8.3 

4.7 

12.1 

.021 

Pointer  On 

2.5 

1.6 

1.6 

.027 

Laser  On 

0.6 

1.6 

0.8 

.022 

Acquisition 

4.4 

3.9 

3.6 

NS 

Fire  Solution:  B 

4.3 

2.4 

5.3 

.032 

Fire  Solution:  S 

2.1 

2.2 

2.6 

NS 

Fire  Decision:  B 

1.3 

1.9 

1.2 

NS 

Fire  Decision:  S 

1.2 

1.4 

0.8 

NS 

Time  to  Fire 

17.1 

12.2 

19.7 

.008 

Fire  Duration 

7.2 

9.4 

8.5 

NS 

Engagement  Time 

24.3 

21.5 

28.2 

NS 

All  of  these  measures  represent  or  Include  times  for  crew  actions.  With 
respect  to  select  time,  the  Category  III  crew  was  significantly  faster:  3.6 
seconds  faster  than  the  Category  II  crew  and  7.4  seconds  faster  than  the 
Category  IV  crew.  For  pointer  on  time,  the  Category  III  and  Category  IV  crews 
were  only  0.01  seconds  apart,  but  the  Category  II  crew  was  significantly 
slower.  For  Time  to  Fire  as  with  select  time,  it  was  the  Category  III  crew 
which  was  fastest.  Laser  on  time  was  the  only  significantly  different  measure 
on  which  the  Category  II  crew  was  fastest. 

Fire  Solution  blinking  times  were  also  significantly  different  as  a 
function  of  crew  Mental  Category.  Mean  times  for  the  Category  III  crew  were 
shorter  than  those  of  either  the  Category  II  crews  or  the  Category  IV  crew, 
but  it  Is  not  clear  how  a  hardware/sof tware  function  such  as  that  would  be 
influenced  by  crew  Mental  Category. 

These  data  do  not  show  the  disadvantage  for  Mental  Category  IV  operators 
that  would  be  expected  on  the  basis  of  earlier  studies.  Although  "those  in 
the  higher  AFQT  categories  ...are  more  likely  to  qualify  for  specialized 
training  in  a  greater  number  of  occupational  areas  and  perform  better  on  the 
job  than  their  low  scoring  peers"  (Army  Field  Circular  FC21-451,  p.  2-5), 
there  is  no  evidence  on  the  basis  of  the  Sgt  York  FOE  I  data  to  show  that 
Category  II  crews  are  superior  to  average  crews.  Only  with  respect  to  laser- 
on  time  was  the  Category  II  crew  superior  and  there  the  difference  was  less 
than  a  second:  The  Category  II  crew  was  1.0  second  faster  than  the  Category 
III  crew  and  0.15  second  faster  than  the  Category  IV  crew.  For  all  other 
measures  on  which  there  was  a  significant  difference,  either  the  Category  III 
crew  or  both  the  Category  III  and  the  Category  IV  crews  were  quicker  than  the 
Category  II  crew. 

There  were  two  unique  squad  leader/gunner  pairings  in  which  both  crew 
members  belonged  to  Mental  Category  II.  The  gunner  was  the  same  in  both 
crews,  but  the  squad  leader  was  not.  The  performance  of  these  two  crews  was 
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compared  and  found  to  be  very  similar.  None  of  the  differences  was  sig¬ 
nificant. 

The  comparisons  just  presented  looked  at  crews  on  which  the  Mental 
Category  of  the  squad  leader  and  the  gunner  matched.  There  was  also  a 
variety  of  mixed  category  crews,  that  is,  crews  in  which  the  Mental  Category 
of  the  squad  leader  and  the  gunner  differed.  As  a  result,  it  was  possible  to 
compare  the  performance  of  crews  in  which  the  squad  leader  was  of  a  higher 
Mental  Category  than  the  gunner  (i.e..  Crews  E,  G,  and  H)  with  the  performance 
of  crews  in  which  the  squad  leader  was  of  a  lower  Mental  Category  than  the 
gunner  (i.e..  Crews  D  and  I).  In  a  comparison  of  all  eleven  segments  of  the 
target  engagement  sequence,  from  select  time  through  total  engagement  time,  no 
significant  differences  were  found.  In  other  words,  crew  performance  was 
essentially  the  same  whether  the  squad  leader  belonged  to  a  higher  or  a  lower 
Mental  Category  than  did  the  gunner.  As  an  example.  Time  to  Fire  was  17.7 
seconds  if  the  squad  leader  was  higher  in  Mental  Category  and  it  was  17.1  if 
the  gunner  was  higher  in  Mental  Category.  Apparently,  at  least  with  the  crew 
members  who  took  part  in  Sgt  York  FOE  I,  the  relative  Mental  Categories  of 
crew  members  did  not  affect  performance. 

Two  other  comparisons  of  note  were  made.  In  one,  the  crews  which  remained 
together  in  stable  working  relationships  were  compared  to  the  crews  in  which 
the  members  were  frequently  interchanged.  The  so  called  stable  crews  might  be 
expected  to  have  an  advantage  over  the  less  stable  crews,  but  there  were  no 
significant  differences  among  them. 

Finally,  crews  in  which  the  squad  leader  had  previous  Sgt  York  experience 
were  compared  with  crews  in  which  the  squad  leaders  had  no  such  previous 
experience.  None  of  the  gunners  who  took  part  in  the  Force-on- Force  Sgt  York 
tests  had  previous  experience  with  Sgt  York.  There  were  significant  differen¬ 
ces  on  three  measures  as  Table  51  shows.  Crews  in  which  squad  leaders  were 
experienced  had  significantly  shorter  laser-on  times,  but  crews  with  Inex¬ 
perienced  squad  leaders  had  shorter  select  and  Time  to  Fire  times.  It  may  be 
that  previous  experience  not  just  with  Sgt  York  but  with  other  similar 
systems  (in  which  laser  operation  was  an  Important  component)  accounts  both 
for  the  quicker  lase  times  found  for  experienced  squad  leaders  and  for  slower 
times  on  those  operations  unique  to  Sgt  York. 

Table  52  presents  a  summary  of  the  differences  observed  in  crew  per¬ 
formance.  That  some  crews  were  faster  than  others  is  neither  surprising  nor 
very  informative,  but  the  observation  that  crews  which  stayed  together 
throughout  the  Force-on-Force  trials  did  not  outperform  the  crews  which 
worked  together  only  occasionally  is  interesting.  Furthermore,  the  fact  that 
crews  whose  members  belonged  to  Mental  Category  II  did  not  consistently  or 
significantly  outperform  Category  III  or  Category  IV  crews  and  that  the 
overall  performance  of  Category  III  crews  was  the  fastest  is  informative. 

Even  in  view  of  the  limited  sample  of  crews  represented  and  the  nature  of  the 
test  environment,  these  outcomes  are  worth  noting. 
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Table  51 

Mean  Times  for  Target  Engagement 
Leader 

Tasks  as  a  Function 

of  Experience  of  Squad 

Task  |  Inexperienced 

SL  I  Experienced  SL 

|  Significance  Level 

(s) 

(s) 

Select  5.9 

9.1 

.025 

Pointer  On  2.4 

2.2 

NS 

Laser  On  1.4 

0.8 

.010 

Acquisition  3.9 

4.4 

NS 

Fire  Solution:  B  3.6 

4.4 

NS 

Fire  Solution:  S  2.5 

2.6 

NS 

Fire  Decision:  B  2.2 

1.3 

NS 

Fire  Decision:  S  1.1 

1.4 

NS 

Time-to-Fire  14.6 

17.9 

.024 

Fire  Duration  9.0 

7.4 

NS 

Engagement  Time  23.6 

25.2 

NS 

Table  52 

Summary  of  Crew  Comparisons 

Comparison  I  Significant  Differences  |  Fastest 

Unique  Crews  (A  through  I) 

Time-to-Fire 

Crews  B  &  I 

Gunner  Mental  Category 

No  Significant 

(11,  1I1A,  IIIB,  IV) 

Differences 

Composite  Crew  Mental 

Select  Time 

Category  IHCrew 

Category  (11,  III,  IV) 

Pointer  On  Time 

Category  IV  Crew 

Laser  On  Time 

Category  II  Crew 

Fire  Solution:  B 

Time  Category  IHCrew 

Time  to  Fire 

Category  IHCrew 

Two  Different  Category  II 

No  Significant 

Crews 

Differences 

Stable  vs.  Changing  Crews 

No  Significant 

Differences 

Experience  of  Squad  Leaders 

Select  Time 

Inexperienced  SL 

Laser  On  Time 

Experienced  SL 

Time  to  Fire 

Inexperienced  SL 
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Training 

Training  and  Target  Decisions 

This  section  presents  information  on  Individual  crew  member  mean  perfor¬ 
mance  times  as  they  relate  to  previous  Sgt  York  experience,  individual 
training  scores,  and  collective  training  outcomes.  Additional  analyses 
Include  crews  with  experienced  squad  leaders  versus  crews  with  inexperienced 
squad  leaders.  Keep  in  mind  the  small  number  of  individuals  involved. 

Table  53  identifies  each  crew  member  in  the  sample  according  to  whether 
or  not  he  had  previous  Sgt  York  experience,  individual  training  scores, 
collective  training  outcomes,  and  mean  performance  time  in  seconds  for  three 
crew  performance  measures  that  could  be  associated  with  specific  individuals: 
selecting/classifying  targets,  pointing  targets,  and  lasing  targets. 


Table  53 

Sgt  York  Player  Personnel:  Previous  Experience  and  Training 


Player  | 

I  Previous  | 

I  Training 

i  Performance  Times 

ID# 

Sgt  York 
Experience 

Ind.  | 

I  Col. 

Select/  | 

Pointer  | 

Laser 

Classify 

On 

On 

X 

X 

X 

(S) 

(S) 

(s) 

0240 

No 

94.9 

Sat . 

5.0 

1.8 

1.5 

1746 

Yes 

NA 

Sat. 

12.0 

2.3 

— 

3753 

Yes 

NA 

Sat. 

9.6 

1.6 

0.3 

4343 

No 

96.7 

Sat. 

5.6 

1.3 

— 

5197 

No 

Fail 

Sat. 

7.3 

3.2 

1.2 

6111 

No 

97.6 

Sat. 

13.7 

2.3 

0.9 

6640 

No 

91.3 

Sat. 

7.6 

3.8 

— 

9089 

Yes 

NA 

Sat . 

5.6 

1.6 

9843 

No 

97.2 

Sat. 

4.7 

2.4 

0.9 

Note.  Ind.  ■  Individual 

Col.  ■  Collective 

Sat.  *  Satisfactory 

NA  «  Not  Applicable;  players  with  previous  Sgt  York 

experience  were  not  included  in  pre-FOE  I 

individual  training. 

Previous  Sgt  York  Experience.  Column  2  of  Table  53  reflects  whether  or 
not  individuals  had  previous  experience  with  the  Sgt  York  fire  unit. 
Individual  crew  members  who  had  been  involved  with  the  Early  Production  Unit 
Test  (EPUT)  and  the  Limited  Test  (LT)  were  classified  as  having  previous  Sgt 
York  experience.  Of  the  30  crew  members  who  participated  in  FOE  I,  six  crew 
members  had  previous  Sgt  York  experience.  During  the  Force-on-Force  phase  of 
FOE  I,  3  of  the  9  operators  had  previous  Sgt  York  experience.  These  persons 
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were  all  squad  leaders.  Soldiers  who  had  previous  Sgt  York  experience  did  not 
participate  in  the  individual  training  course  that  preceded  FOE  I.  Crew 
members  identified  in  this  table  participated  in  the  trials  and  engagements 
from  which  the  performance  data  base  presented  earlier  (Tables  3,  4,  and  5) 
was  generated. 

Previous  Sgt  York  experience  and  mean  Select  and  Classify  times.  Gunner 
and  squad  leader  individual  mean  select  times  were  compared  for  the  first  crew 
activity,  Select  and  Classify  targets.  Results  indicated  that  there  was  a 
significant  difference  (p*».007).  The  two  crew  members  who  had  the  fastest 
performance  were  Gunner  #0240  (5.0  seconds  mean),  and  Squad  Leader  #9843  (4.7 
seconds  mean).  Neither  crew  member  had  previous  experience.  For  this  limited 
sample,  having  no  previous  experience  with  the  fire  unit  was  found  to  be  ad¬ 
vantageous  in  selecting  (and  pointing)  a  target.  Experienced  crew  members  did 
not  take  part  in  FOE  I  individual  training.  The  performance  times  of  the 
inexperienced  crew  members  for  Select  and  Classify  targets  may  have  been  en¬ 
hanced  by  their  recent  individual  training. 

Previous  Sgt  York  experience  and  mean  Pointer  On  times.  Another  com¬ 
parison  of  crew  mean  performance  times,  for  crew  members  doing  the  pointer  on 
task,  was  made.  Crew  members  varied  significantly  on  the  amount  of  time  it 
took  them  to  point  a  target  (p“.003).  Four  crew  members  had  mean  pointer  on 
times  of  under  two  seconds  (Gunner  #0240,  1.8  seconds;  Squad  Leader  #3753,  1.6 
seconds;  Squad  Leader  #4343,  1.3  seconds;  and  Squad  Leader  #9089,  1.6 
seconds).  Two  of  these  four  crew  members  had  previous  Sgt  York  experience. 
Crewmen's  speed  in  pointing  a  target  was  not  associated  with  previous 
experience  or  lack  of  experience  with  the  fire  unit. 

Crews  with  experienced  squad  leaders  versus  crews  with  inexperienced 
squad  leaders.  Crew  members  were  sorted  into  those  with  experienced  squad 
leaders  and  those  with  inexperienced  squad  leaders  to  see  if  this  factor  made 
a  slgnf leant  difference  in  performance.  These  two  groups  of  crews  were 
compared  on  mean  times  for  Select  and  Classify,  Pointer  On,  Laser  On,  Acquire, 
Fire  Solution  Blinking,  Fire  Solution  Solid,  Fire  Decision  Blinking,  Fire 
Decision  Solid,  Time  to  Fire,  Fire  Duration,  and  Engagement  Time.  Significant 
differences  were  found  for  Select  and  Classify,  Laser  On,  and  Time  to  Fire. 

Crews  with  experienced  squad  leaders  were  found  to  be  significantly 
different  from  crews  with  inexperienced  squad  leaders  (p*.025)  in  performing 
the  Select  and  Classify  activity.  The  mean  time  it  took  a  crew  with  an  inex¬ 
perienced  squad  leader  to  select  and  classify  a  target  was  3.2  seconds  less 
than  for  a  crew  who  had  an  experienced  squad  leader  (see  Table  54). 
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Table  54 


Crews  With  Experienced  Versus  Crews  With  Inexperienced  Squad  Leaders,  Select 
and  Classify  Target 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(8) 

Crews  With  Experienced  Squad  Leader 

133 

9.1 

11.4 

Crews  With  Inexperienced  Squad  Leader 

88 

5.9 

8.5 

Ordinarily,  significantly  quicker  mean  performance  times  would  not  be 
expected  from  novice  squad  leaders  without  previous  fire  unit  experience. 
Perhaps  the  more  recent  individual  training  given  to  inexperienced  squad 
leaders  improved  their  performance  in  selecting  and  classifying  targets,  and 
they  may  have  coached  the  gunner.  Lack  of  recent  individual  training  for 
experienced  squad  leaders  in  this  content  area  may  have  contributed  to  their 
relatively  poorer  performance. 

Laser  on  was  found  to  be  significantly  different  (p**.010)  between  crews 
with  experienced  and  crews  with  inexperienced  squad  leaders.  The  finding 
Indicates  that  crews  with  experienced  squad  leaders  lased  targets  about  a  half 
second  more  quickly  than  crews  with  Inexperienced  squad  leaders  (See  Table 
55).  Apparently,  previous  squad  leader  experience  with  the  fire  unit  was 
beneficial  in  reducing  mean  time  to  perform  the  laser  pointing  task.  Since 
all  but  one  lasing  was  performed  by  gunners,  it  was  particularly  notable  that 
squad  leader  experience  made  a  difference  in  performance.  Previous  experience 
with  laser-using  systems  may  also  have  reduced  mean  lase  times. 


Table  55 

Crews  With  Experienced  Versus  Crews  With  Inexperienced  Squad  Leaders,  Laser 
On 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(8) 

Crews  With  Experienced  Squad  Leader 

32 

0.8 

0.7 

Crews  With  Inexperienced  Squad  Leader 

34 

1.4 

1.0 

Time  to  Fire  was  found  to  be  significantly  different  (p*.024)  between 
crews  with  experienced  versus  crews  with  inexperienced  squad  leaders.  The 
crew  decision  function.  Time  to  Fire,  represents  the  amount  of  time  it  took  a 
gunner  to  depress  the  trigger  after  a  firing  cue  appeared.  Gunners  with 
inexperienced  squad  leaders  were  able  to  fire  3.3  seconds  more  quickly  (mean 
performance  time)  than  gunners  with  experienced  squad  leaders  (see  Table  56). 
It  is  difficult  to  understand  why  a  gunner  with  an  experienced  squad  leader 
would  fire  less  quickly  than  a  crew  with  an  inexperienced  squad  leader.  As 
previously  mentioned,  crewmen  with  previous  Sgt  York  experience  did  not 
receive  individual  training.  Perhaps  the  additional  training  and  the 
experience  combined  would  have  reversed  these  findings,  or  perhaps  recent 


143 


training  was  more  successful  at  overcoming  negative  transfer  from  experience 
with  systems  other  than  Sgt  York. 


Table  56 

Crews  With  Experienced  Versus  Crews  With  Inexperienced  Squad  Leaders,  Time  to 
Fire 


ENGAGEMENTS 

MEAN 

S.D. 

(n) 

(8) 

(8) 

Crews  With  Experienced  Squad  Leaders 

162 

17.9 

12.4 

Crews  With  Inexperienced  Squad  Leaders 

109 

14.6 

10.8 

Individual  Training.  Individual  and  collective  training  scores  and 
ratings  have  been  included  in  Column  3  (Individual  training)  and  Column  4 
(collective  training)  of  Table  53. 


Individual  training  was  conducted  at  Fort  Bliss,  TX,  between  15  October 
and  21  December  1984.  The  Program  of  Instruction  (POI)  was  six  weeks  and 
three  days.  The  POI  had  been  modified  and  reduced  from  an  original  11  weeks 
and  two  days  to  its  shorter  length.  Limited  training  days  were  available 
before  the  initiation  of  FOE  I,  and  this  prompted  the  modification  to  the  POI. 

As  previously  mentioned  pertaining  to  the  column  "Previous  Sgt  York 
Experience,"  not  all  crewmen  received  individual  training.  Six  crewmen  had 
previous  Sgt  York  experience  and  had  been  trained  by  Ford  Aerospace  and 
Communications  Corporation  (FACC).  Three  of  these  individuals  participated 
in  the  Force-on-Force  trials  and  are  identified  in  Column  3  by  "Not  Ap¬ 
plicable"  because  they  were  not  included  in  the  pre-FOE  I  individual  train¬ 
ing. 

Individual  training  scores  and  mean  Select  and  Classify  times.  Gunner  and 
squad  leader  mean  performance  times  were  compared  for  Select  and  Classify 
targets,  a  crew  activity.  There  was  a  significant  difference  (p=.007);  the 
two  crew  members  having  the  highest  level  of  performance  were  Gunner  #0240, 

5.0  seconds  mean;  and  Squad  Leader  #9843,  4.7  seconds  mean.  When  these  two 
crew  members  were  compared  on  individual  training  scores,  there  was  a 
difference  of  2.3  points  in  scoring  between  the  crewmen.  Figure  39  shows  the 
individual  training  scores  by  the  mean  select/classify  times  for  Individual 
crew  members.  Higher  individual  training  scores  are  not  necessarily  as¬ 
sociated  with  higher  performance  (quicker  times)  in  selecting  and  classifying 
a  target. 


Individual  training  scores  and  mean  Pointer  On  times.  Crew  members  were 
compared  on  their  mean  performance  times  in  pointing  targets  (pointer  on). 

Mean  times  on  this  task  were  found  to  be  significantly  different  (p-.003). 

The  fastest  performing  crew  members  with  mean  time  under  2.0  seconds  were 
Gunner  #0240,  1.8  seconds;  Squad  Leader  #3753,  1.6  seconds;  Squad  Leader 
#4343,  1.3  seconds;  and  Squad  Leader  #9089,  1.6  seconds.  Two  of  these  crewmen 
had  individual  training  scores  of  94.9  and  96.7.  The  other  two  crewmen  did 
not  receive  pre-FOE  I  individual  training  since  they  had  previous  Sgt  York 


experience.  This  is  an  indication  that  the  task  of  pointing  a  target  may  be 
taught  in  the  classroom  or  during  on-the-job  training  (OJT).  Both  methods  of 
instruction  were  just  as  proficient  in  allowing  the  four  crewmen  to  achieve  a 
high  level  of  performance  for  the  pointer  on  task.  Crew  members  used  in  this 
sample  were  compared  for  mean  pointer  on  performance  times  by  individual 
training  scores.  Figure  40  illustrates  that  a  higher  individual  training 
score  does  not  indicate  that  the  crew  member  will  have  a  faster  pointer  on 
performance. 

Collective  Training.  Collective  training  was  conducted  at  Fort  Bliss 
between  31  December  1984  and  15  February  1985.  Those  individuals  who  were 
certified  as  satisfactorily  completing  collective  training  were  identified 
with  a  "Sat."  in  Table  53.  It  was  not  possible  to  differentiate  among  crew 
members  regarding  their  certification  or  lack  of  certification  for  the 
completion  of  collective  training  since  all  crew  members  in  the  sample 
received  a  designation  of  satisfactory.  Collective  training,  as  well  as 
individual  training,  did  not  seem  to  influence  the  ability  of  any  crew  member 
to  perform  the  laser  on  task  more  quickly. 

Trends  among  training  scores,  previous  Sgt  York  experience,  designations, 
and  performance  times  were  net  found.  This  may  have  been  due  to  the  limited 
sample  size.  Uith  a  larger  sample,  it  might  have  been  possible  to  establish 
trends.  However,  it  is  not  unusual  to  find  a  lack  of  supporting  data  for 
transfer  of  training  from  the  classroom  to  the  field.  Data  presented  on 
experience  and  training  are  not  sufficient  to  assess  selection  criteria  for 
training  cutoff  scores,  collective  training  designation,  or  amount  of 
training. 

Training  cutoff  scores  used  as  a  criterion  for  selection  or  other 
established  criteria  for  collective  training  were  not  appropriate  for 
selecting  crew  members  due  to  the  contingencies  of  the  training  environment. 
Because  of  the  accelerated  fielding  schedule,  the  fire  units  were  retrofitted 
for  software  and  hardware  during  training  and  testing.  Crew  members  were  not 
familiar  with  the  changes,  and  the  impact  of  the  retrofitting  was  not  taken 
into  account  for  training  scoring  and  designation  purposes. 

Other  factors  which  influenced  the  reliability  of  the  training  are 
associated  with  a  transfer  of  training  from  the  VULCAN  to  the  Sgt  York  fire 
unit.  Squad  leaders  previously  trained  and  experienced  with  the  VULCAN  were 
found  to  use  the  laser/optics  instead  of  the  radar  during  training  sessions. 
The  preference  for  laser/optics  was  identified  for  situations  where  there  was 
no  tactical  necessity  for  the  use  of  that  system  mode.  Since  Radar  Auto  mode 
had  the  highest  level  of  automation  and  was  the  most  efficient  mode  in  the 
completion  of  a  target  engagement  sequence,  it  would  be  counterproductive  to 
use  laser/optics  when  they  were  not  needed.  Overall,  squad  leaders  with 
previous  VULCAN  training  and  experience  distrusted  the  Sgt  York  fire  unit. 
Previous  VULCAN  training  may  have  been  detrimental  to  Sgt  York  crew  member 
performance  and  appeared  to  have  negatively  impacted  the  Sgt  York  training. 

Use  of  Initiative.  Squad  leaders  were  taught  how  to  read  maps,  but  they 
had  a  difficult  time  generalizing  this  knowledge  into  terrain  reading.  This 
skill  was  required  at  Hunter-Liggett  in  order  to  execute  the  decentralized 
mission  of  the  autonomous  fire  units.  The  resulting  behavior  appeared  to  be 
due  to  a  lack  of  confidence  to  execute  a  decentralized  mission.  However,  on 
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the  airland  battlefield,  crew  members  would  be  required  to  6how  initiative  and 
use  common  sense  to  achieve  their  commander's  objectives.  To  operate  as  in¬ 
dependent  elements  would  be  necessary  (Babbitt,  Seven,  Lyons,  &  Sparks,  1986). 
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Mean 


Future  FAADS  Soldier-System  Performance  Models 


Quantitative  Prediction  Models 

In  the  Introduction,  the  point  was  made  that  a  desirable  tool  for  the 
future  design  of  FAADS  is  quantitative  system-human  performance  computer 
simulation  models.  One  major  purpose  of  this  report  is  to  provide  concepts 
and  data  that  might  be  useful  for  that  goal.  The  purpose  of  this  chapter  is 
to  see  what  concepts  and  data  presented  in  the  chapters  on  human  performance 
data  and  on  research  findings  could  be  applied  to  future  FAADS  soldier-system 
performance  models. 

In  the  Introduction,  some  fundamental  requirements  for  modeling  were 
noted.  They  include  at  least  three  sequential  areas: 

•  The  FAADS  to  be  modeled  must  be  described  including  major  system 
functions  and  tasks  both  with  respect  to  the  system,  subsystems,  crew 
and  human  operator  tasks. 

•  For  system  functions  and  tasks,  parameter  identification  must  be  made 
to  insure  that  appropriate  parameters  are  included  in  the  model. 

•  For  the  model  parameters,  performance  data  must  be  available  either 
from  existing  empirical  sources  or  data  estimation. 

A  fourth  general  requirement,  selection  of  the  best  computer  language  and 
software,  will  not  be  discussed  here  because  that  is  more  of  a  problem  of 
specific  choice  of  how  the  soldier-system  is  to  be  simulated  rather  than  what 
is  to  be  simulated. 


System  Description 

The  chapter  on  human  performance  data  opened  with  a  detailed  discussion 
of  the  FAADS  target  engagement  sequence  first  in  the  most  generic  sense,  then 
with  examples  of  variations  expressed  by  the  ADATS,  Liberty,  Rapier,  Paladin, 
and  Sgt  York  systems.  It  is  this  kind  of  system  description  that  must  form 
the  basis  for  quantitative  prediction  models. 

Further,  there  will  always  be  a  question  as  to  the  appropriate  level  of 
detail  to  which  the  system  description  must  be  taken.  One  deciding  factor  in 
determining  the  appropriate  level  always  is  the  kind  of  design  question  being 
asked.  If,  for  example,  there  is  some  question  about  target  designation,  then 
the  model  must  be  able  to  focus  on  that  task  in  sufficient  detail.  This  was 
the  area  of  interest  in  the  MOPADS  model  (Polito  &  Laughery,  1984),  as  noted 
in  the  Introduction.  On  the  other  hand,  if  the  question  is  to  predict  if 
system  response  times  are  being  achieved,  a  much  different  and  far  more  global 
level  of  system  description  must  be  simulated  and  modeled. 

It  is  hoped  that  the  analysis  presented  in  the  chapter  on  human  per¬ 
formance  data  will  be  helpful  in  any  specific  future  FAADS  modeling  effort. 
While  there  may  be  many  specific  subsystem  differences,  the  sequences  shown 
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in  the  chapter  on  human  performance  data  must  be  represented  in  the  model  in 
some  form  unless,  of  course,  some  radically  new  technology  appears  to  6olve 
forward  area  air  defense  problems. 

For  many  purposes  in  design  -  and  not  just  for  the  development  of 
performance  models  -  traditional  systems  engineering  functional  flow  analyses 
can  serve  at  several  levels  as  the  basis  for  the  development  of  simulated 
system  descriptions.  In  some  system  designs,  these  analyses  are  performed, 
but  in  many  cases  they  are  not.  It  is  critically  important  to  specify  at 
several  levels  of  detail  what  functions  the  system  must  perform  apart  from  the 
question  of  how  the  functions  are  done  (i.e.,  the  actual  mechanization).  It 
is  Impossible  to  develop  an  acceptable  quantitative  prediction  model  without 
that  kind  of  analysis  as  the  foundation  for  the  model. 


Parameter  Identification 


The  chapter  on  research  findings  offers  a  number  of  suggestions  for 
necessary  variables  and  parameters  that  might  be  included  in  a  future  FAADS 
soldier-system  performance  model.  One  of  the  purposes  of  the  analyses  in  that 
chapter  was  to  investigate  what  some  of  the  critical  parameters  might  be. 

That  is,  what  conditions  in  forward  area  air  defense  might  be  expected  to 
influence  individual,  crew,  subsystem,  and  system  performance?  It  is 
important  to  note  that  the  findings  are  suggestive  only.  The  fact  that  a 
variable  was  not  shown  to  be  effective  in  these  data  does  not  automatically 
exclude  it  from  consideration  in  future  model  development. 

In  summary  of  the  research  findings,  the  following  comments  could  be  made 
with  respect  to  model  development: 

•  It  is  useful  to  have  some  quantitative  estimate  of  how  closely  the 
system  reaches,  or  falls  short  of,  system  response  time  requirements. 

Periodic  checks  can  be  made  with  the  model  as  design  progresses  and  subsys¬ 
tems  are  revised  and/or  changed. 

•  If  the  proposed  system  has  multiple  modes  of  operation,  it  seems 
imperative  that  quantitative  predictions  be  available  on  how  well  those  modes 
will  be  used  and  above  all  how  fast  performance  Is  predicted  to  be.  Multiple 
modes  may  In  fact  impede  successful  system  performance. 

•  Where  multiple  fire  units  are  involved,  there  may  be  important  dif¬ 
ferences  in  performance  among  them.  Potential  variability  should  be  inves¬ 
tigated  in  the  model.  Whether  the  fire  units  are  operating  independently  or 
in  a  command  and  control  network  will  also  be  a  critical  modeling  considera¬ 
tion. 


•  The  model  should  be  sufficiently  rich  so  as  to  explore  possible 
variations  in  tactical  performance.  For  example,  the  results  found  here 
suggest  that  processing  times  change  when  the  unit  is  fixed  or  mobile.  But 
the  changes  may  only  be  of  interest  in  subsystem  tasks.  On  the  other  hand, 
for  a  critical  system  measure  -  Time  to  Fire  -  significant  differences  were 
found  for  roeery-wing  tactics,  ECM  condition,  time  of  day,  and  target  range 
at  first  appearance. 
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Some  of  the  variables  and  parameters  looked  at  here  did  not  seem  to  make 
much  difference  in  performance.  This  was  true  of  (1)  scenario  type,  (2) 
comparisons  of  gunner  and  squad  leader  performance,  (3)  the  use  of  MOPP  gear, 
and  (4)  target  workload.  But  they  are  not  necessarily  to  be  ignored  in  future 
models  because: 

•  The  particular  scenarios  exercised  may  have  not  been  different  enough 
to  make  any  particular  impact. 

•  In  crew  performance  models,  it  is  to  be  expected  that  distributions 
of  performance  data  will  be  used  for  different  crew  members. 

•  In  this  instance,  the  MOPP  gear  simply  was  not  worn  long  enough  to 
have  an  effect  upon  performance. 

•  Target  workload  had  no  apparent  impact  because  target  prioritization 
was  performed  by  automatic  decision  aids. 

In  previous  model  developments,  it  has  been  difficult  to  include  certain 
parameters  associated  with  crew  characteristics.  An  example  of  one  such 
variable  is  "mental  category"  or,  using  the  new  terminology,  "test  score 
category."  With  respect  to  this  variable  and  crew  mix,  the  Chapter  IV  finding 
was  that  Time  to  Fire  was  influenced  by  various  crew  mixes.  Related  to  this 
parameter  is  the  impact  of  past  crew  member  experience.  Here,  one  finds  the 
odd  result  chat  crews  with  experienced  squad  leaders  were  slower  on  Time  to 
Fire  than  crews  with  inexperienced  squad  leaders.  Whatever  direction,  crew 
mix  and  individual  crew  member  experience  are  variables  for  modeling. 

Finally,  it  seems  very  reasonable  that  training  parameters  should  somehow 
be  candidates  for  inclusion  in  performance  models.  It  is  probable,  however, 
that  "training"  variables  per  se  are  not  of  direct  model  use,  but  rather  must 
be  translated  to  levels  of  performance  produced  by  training.  This  would  mean, 
for  example,  using  different  soldier  performance  data  distributions  to  reflect 
various  stages  of  human  learning. 


Data  Distributions 

One  of  the  major  purposes  of  this  report  was  to  extract  and  present 
performance  data  distributions  over  a  wide  variety  of  conditions  appropriate 
to  the  forward  area  air  defense  problems.  These  data  could  be  used  for  a 
variety  of  design  purposes,  but  one  purpose  in  particular  is  for  performance 
models  for  future  FAADS.  Such  data  and  their  distributions  have  been 
presented  in  the  discussions  of  the  individual  research  findings;  means  and 
standard  deviations,  as  well  as  actual  frequency  distributions  where  ap¬ 
propriate,  have  been  presented. 

The  principal  data  distributions,  however,  were  shown  in  Tables  3  and  5. 
For  271  complete  target  engagement  sequences,  time  data  are  given  for  11  basic 
system  and  task  functions  by  each  individual  TES.  Where  similar  subsystem 
functions  appear  in  future  FAADS  and  a  system  model  is  developed,  the  data 
presented  in  this  report  may  be  considered  for  model  data  distributions.  It 
is  hoped  that  such  data  will  find  extensive  use  as  human  performance  models 
are  developed  for  future  FAADS. 
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Follow-On  Evaluation  Procedures 


Follow-on  evaluation  procedures  for  operational  test  and  evaluation 
(OT&E)  are  viewed  from  a  perspective  of  opportunities  and  possibilities  of 
what  could  be  accomplished  if  MANPRINT  methodology  actually  had  been  applied 
during  FOE  I.  The  specific  objectives  of  this  chapter  are  multidimensional, 
and  are  designed  to  heighten  awareness  of  lessons  relearned,  to  improve  the 
weapon  system  acquisition  process  and  system  designs  of  the  future,  and  to 
suggest  what  could  be  accomplished  for  future  OT&E.  Discussion  of  follow-on 
evaluation  procedures  covers  topic  areas  for  MANPRINT  implications,  test 
planning  with  an  emphasis  on  preplanning  activities,  data  analyses  and  inter¬ 
pretation,  realistic  test  environment,  personnel  characteristics  and  training 
and  performance  measures,  and  dissemination  of  Information  from  OT&E. 


MANPRINT  Implications 

The  integration  of  MANPRINT  procedures  into  OT&E,  as  the  next  iteration 
from  the  previous  manpower,  personnel,  and  training  OT&E  used  by  human 
factors  engineers  in  operational  testing,  will  be  considered.  The  total 
MANPRINT  methodology  for  an  integrated  approach  to  the  design  of  equipment 
which  includes  procedures  for  Imposing  manpower  and  personnel  integration 
within  the  full  acquisition  process  (AR  602-2,  1987)  will  not  be  dealt  with 
here. 

Because  the  complexity  of  Army  systems  has  been  increasing  over  time, 
soldiers  are  now  required  to  be  capable  of  performing  tasks  with  significantly 
higher  cognitive  demands.  Complex  tasking  for  complex  systems  has  compromised 
soldier-system  performance  and  force  effectiveness.  For  example,  complex 
tasks  in  the  Sgt  York  fire  unit  were  identified  for  operators  for  tasks 
associated  with  selection  and  use  of  the  multiple  system  modes  during  the 
target  engagement  sequence.  Other  complex  tasks  were  identified  during  FOE  I 
for  the  maintainers  regarding  bit  isolation.  Responsibility  for  resolving  the 
problem  of  complex  tasking  for  complex  systems  should  be  a  joint  effort 
between  the  Army  Materiel  Command  and  the  private  sector.  They  should  perform 
analyses  which  illuminate  tradeoffs  between  design  alternatives,  manpower, 
personnel,  and  training. 

MANPRINT  involves  a  higher  level,  and  more  integrated  level,  of  analy¬ 
ses  than  previously  performed  during  OT&E.  There  are  no  standard  methods  for 
applying  MANPRINT,  yet  former  OT&E  test  planning  and  implementation  will  no 
longer  be  viable  because  of  the  MANPRINT  initiative.  Structurally  sound  MAN- 
PRINT  methodology  should  include  task  analyses  that  are  performed  to  identify 
critical  tasks  for  overall  system  performance.  These  critical  tasks  must  be 
viewed  in  terms  of  overall  system  performance  as  it  relates  to  ergonomic  and 
cognitive  skills  for  the  population  of  soldiers  who  are  available,  and  who  are 
projected  to  be  available  over  the  life  cycle  of  the  weapon  system.  Outcomes 
from  the  measurement  are  then  designed  to  answer  questions  such  as:  What  is 
the  quality  of  the  personnel,  will  there  be  potential  changes  in  the  quality 
and  quantity  of  the  personnel,  how  can  training  and  system  design  reduce 
stress  on  the  soldier  and  help  to  attain  system  performance  goals? 


Test  design  plans  of  the  future  will  not  be  able  to  get  away  from  the 
requirement  for  more  and  more  measurement.  In  order  to  Investigate  the 
MANPRINT  areas,  HFE  personnel  activities  associated  with  data  collection, 
analyses,  and  dissemination  of  information  will  be  expanded.  To  illustrate 
the  type  of  expansion  required  for  the  implementation  of  MANPRINT  data 
collection  techniques,  critical  task  measurement  has  been  refined.  In  a 
previous  test  where  MANPRINT  techniques  were  applied  (Remotely  Piloted 
Vehicle) ,  critical  task  crew  performance  measures  were  established  through 
critical  task  assessment  interviews,  on-site  observation,  structured  inter¬ 
views  covering  the  test  data  requirements,  and  comment  and  opinion  data  from 
test  participants.  Critical  tasks  identified  and  integrated  into  the  multiple 
data  collection  techniques  all  had  timed  crew  performance  measures  amenable  to 
timed  data  collection  conditions  which  were  tied  into  the  test  instrumenta¬ 
tion.  Criteria  were  applied  to  the  selection  and  retention  of  critical  crew 
performance  tasks.  Without  criteria,  it  would  have  been  impossible  to  assess 
the  tasks  and  interpret  the  data  with  any  assurance  of  validity.  During  Sgt 
York  FOE  I,  the  test  plan  did  not  identify  critical  tasks  (Babbitt,  1987; 
Seven,  1987).  Criteria  were  not  established  prior  to  or  during  analyses  of 
the  test  results.  The  measurement  process  was  constrained  and  forced  into  a 
format  of  describing  the  data,  but  the  element  of  prediction  based  on  critical 
tasks  and  criteria  had  no  solid  methodological  foundation.  Generalization  of 
the  test  results  was,  therefore,  tenuous. 

The  design  of  the  test  plan  has  increased  in  importance  due  to  the  MAN- 
PRINT  requirements.  In  order  to  develop  a  test  plan  with  multiple  measures  to 
provide  answers  for  the  six  MANPRINT  areas  (human  factors,  manpower,  training, 
personnel,  system  safety,  and  health  hazards),  time  allotted  for  preplanning 
of  the  test  must  be  expanded.  To  provide  an  example  of  the  level  of  effort 
that  is  now  needed  to  address  MANPRINT  in  preplanning  and  conducting  the  test, 
and  post-test  activities,  a  task  listing  was  developed  as  a  guide  for 
implementing  the  MANPRINT  initiative,  as  shown  in  Table  57  (Krohn,  1986). 

Resulting  documentation  from  the  accomplishments  of  the  task  listing  are 
identified  in  Table  57.  As  can  be  seen  by  a  review  of  the  pretest  activities, 
test  conduct  activities,  and  post-test  activities,  more  technical  manpower 
loading  is  required  for  test  personnel  to  complete  the  pretest  activities  than 
to  conduct  the  test  or  the  post-test  activities.  There  is  a  profound  labor 
shift  among  OT&E  test  activities  and  professional  labor  hours  to  plan  and 
conduct  tests  now  compared  to  previous  years.  MANPRINT  OT&E  has  become  a  more 
labor-intensive  endeavor  than  in  previous  years.  The  shift  in  pretest 
activities,  the  increased  data  collection  and  analyses,  and  the  new  approaches 
to  reporting  and  disseminating  information  have  had  a  significant  impact  on 
budget  preparation  activities  and  funding  requirements.  Allocating  resources 
within  the  test  plan,  and  obtaining  funding  necessary  to  achieve  acceptable 
integration  of  data  and  outcomes  promised  by  MANPRINT  efforts  have  the 
potential  to  become  productive  efforts.  OT&E  planning  and  implementation  of 
tests  conducted  using  new  and  evolving  techniques  have  been  found  to  increase 
test  budgets  by  more  than  90%  over  what  would  have  been  incurred  prior  to  the 
MANPRINT  initiative.  In  the  name  of  the  soldier-machine  interface,  where  all 
the  test  community  effort  counts,  the  question  still  remains  as  to  whether  the 
federal  government  has  the  resources  to  fund  MANPRINT  OT&E.  Obviously, 
without  adherence  to  administrative  procedures  already  in  place,  and  without 
appropriate  funding,  the  OT&E-MANPRINT  marriage  could  easily  become  unhinged. 
Adherence  to  MANPRINT  would  retain  checkpoints  embedded  throughout  the  weapon 
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Table  57 


Major  Operational  Test  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems 


RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


—  — . 

PRETEST 


1.  Coordination  meetings  with  OTEA  POC 
and  Test  Directorate. 

1. 

None  (May  require  more  than  one 
meeting.) 

2.  Develop  MANPRINT  test  issues  and 
criteria. 

2. 

List  of  test  Issues  and  criteria  in 
OTEA  format  for  Inclusion  in  draft 
Test  Design  Plan. 

3.  Review  previous  test  documentation 
and  findings. 

3. 

a.  List  of  MANPRINT  areas 
requiring  full  investigation 
or  additional  data. 

b.  Summary  included  in  ARI 
MANPRINT  test  report. 

4.  Plan  human  performance  measures  side 
tests  and  critical  task  assessment  (CTA). 

4. 

Draft  experimental  or  quasi - 
experimental  design  plans  for 
inclusion  in  draft  Detail  Test 
Design  Plan. 

5.  Plan  training  assessment. 

5. 

Training  Assessment  Plan. 

6.  Meet  with  OTEA  to  Integrate  human 
performance  measures  and  automated 
data  processing  efforts. 

6. 

Final  draft  of  MANPRINT  sections 
of  the  Test  Design  Plan  and 

Detail  Test  Plan.  (May  require 
more  than  one  meeting.) 

7.  Conduct  test  system  familiarization: 

7. 

Site  visit  report: 

a.  Visit  ongoing  testing  (e.g.y 
developmental  testing), 

a.  Interview  data, 

b.  Photo  documentation. 

b.  Visit  pretest  player  training. 

c.  Visit  predecessor  systems  operations, 

d.  Visit  system's  manufacturing  facility. 
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Table  57 


Major  Operational  Test  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems  (Cont'd.) 


RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


PRETEST  (confd) 


8.  Conduct  pretest  training  evaluation  at 
player  training  slt^: 

a.  Attend  classroom  sessions, 

b.  Attend  field  demonstrations, 

c.  Review  training  documentation, 

d.  Administer  Interviews, 

e.  Administer  questionnaires  and 
pretest  training  test. 

9.  Validate  selection  of  critical  tasks 
for  CTA: 

a.  Visit  and  review  player  training, 

b.  Visit  and  review  ongoing  testing, 

c.  Interview  test  system  subject 
matter  experts. 


8 .  Site  visit  report  and  later  the 
final  AR1  training  assessment 
report  or  sections  In  the  final 
ARI  MANPRINT  test  report. 


9.  List  of  critical  tasks  for  which 
performance  measures  will  be 
collected. 


10.  Prepare  Detail  MANPRINT  Data  Collection  10.  Draft  MANPRINT  human  factors 
Plan:  assessment  plan  and  data 

collection  materials. 

a.  Overview  of  plan, 

b.  MANPRINT  tasks, 

c.  Test  data  requirements, 

d.  Schedules, 

e.  Detail  data  collection  plans, 

f.  Staffing, 

g.  ADP  requirements. 


11.  Prepare  detail  data  collection  materials: 

a.  Interviews, 

b.  Checklists, 

c.  Observations  log, 

d.  Daily  debrief  and  comments  log, 

e.  Comment  code  taxonomy. 

12.  Review  with  ARI  OTEA  designates  the 
detailed  MANPRINT  data  collection  plan 
and  data  collection  materials. 


11.  Draft  MANPRINT  human  factors 
assessment  plan  and  data 
collection  materials. 


12.  Final  MANPRINT  human  factors 
assessment  plan  and  data 
collection  materials. 
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Table  57 


Major  Operational  Test  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems  (Cont'd.) 

RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


PRETEST  (cont'd) 


13.  Integrate  MANPRINT  data  collection 
plan  and  human  performance  measures 

Into  automated  data  base. 

13. 

Draft  ADP  requirements,  input 
output  formats,  and  review  of 
system  performance  measures. 

14.  Prepare  detail  experimental,  quasi* 
experimental,  and/or  critial  task 
assessment  methodology. 

14. 

Experimental  design  and  CTA 
performance  measure  worksheets. 

15.  Demonstrate  ADP  Integration. 

15. 

Fictitious  data  files  for  ADP 
operations  check. 

16.  Attend  operational  test  readiness 
reviews  (OTRR). 

16. 

None.  (May  require  more  than 
one  meeting.) 

17.  Prepare  test  data  collector  training 
for  MANPRINT  effort.  Give  data 
collector  training  related  to 

MANPRINT. 

17. 

a.  List  data  collection  Items 
to  be  collected  by  data 
collectors. 

b.  Lists  of  operational  defi¬ 
nitions  for  performance 
measures  to  be  collected 
by  data  collectors. 

c.  Examples  of  MANPRINT  data 
requirements  and  crew 
behaviors  regarding  the 
beginning  and  ending  of  the 
operational  definitions. 

18.  Obtain  test  player  demographic, 
experience,  training,  and  aptitude 
data  from  the  soldier  support  center 
or  other  sources  as  appropriate. 

19.  Manage  MANPRINT  specialist  staffing. 
Prepare  budget  and  expense  data. 

19. 

Budget  and  expense  data. 

♦20.  Prepare  MANPRINT  system  baseline 
prediction  assessment  plan. 

20. 

MANPRINT  system  baseline 
predictions  validation  plan. 

♦Potential  new  area  of  Investigation.  During  the  design  phase  of  new  system  development, 
MANPRINT  MPT  predictions  will  be  made  concerning  manpower  quantity  and  quality. 
During  operational  testing  the  predictions  for  each  MOS  assigned  to  the  system  should  be 
validated. 
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Table  57 


Major  Operational  Test  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems  (Cont'd.) 


RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


TEST  CONDUCT 


1.  Attend  test  site  collective  training. 

2.  Attend  pilot  testing: 

a.  Verify  data  collection  procedures 
for  performance  measures, 

b.  Collect  Interview  data, 

c.  Collect  daily  debrief  and  comment 
data, 

d.  Observe  field  operations, 

e.  Attend  daily  data  analysis  group 
DAG  meetings, 

f.  Refine  data  collection  materials* 


1.  None. 

2.  a.  Completed  data  collection 

forms  and  comment  listing. 

b.  Preliminary  ADP  output. 

c.  Revisions  to  data  collection 
materials  if  necessary. 


3.  Attend  full  test:  3. 

a.  Insure  proper  CTA  performance 
measure  data  collection, 

b.  Administer  data  collection  materials, 

(1)  Interviews 

(2)  Checklists 

(3)  Questionnaires 


a.  Completed  data  collection 
forms. 

b.  Preliminary  ADP  raw  data 
output. 

c.  Preliminary  safety  and  health 
hazards  assessment  report  as 
necessary. 


c.  Perform  safety  assessment  of  safety / 
health  hazard  problem  areas  as 
necessary, 


d.  Data  file  output  summary 
formats  for  findings  to 
date. 


d.  Observe  field  operations, 

e.  Attend  daily  player  debriefs, 


e.  Preliminary  listing  and 
description  of  MANPR1NT 
findings. 


f.  Attend  RAM,  SYSTEM  PERFORMANCE, 
and  MANPR1NT  DAG. 


f.  Outlines  for  OTEA  test 
report  sections  and  the 
ARI  MANPRINT  report. 
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Table  57 


Major  Operational  Test  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems  (Cont'd.) 


RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


TEST  CONDUCT  (confd) 

g.  Collect  and  categorize  opinion  and 
comment  data, 

h.  Collect  manpower  requirements  data 
from  field  observations  and  RAM 
incident  data  forms, 

1.  Administer  training  retention  tests 
and  obtain  comments  and  opinions 
from  players  concerning  training, 

j.  Insure  proper  ADP  data  file  develop¬ 
ment.  Review  cumulative  data  to 
date. 

k.  Collect  instrumented  measures  of 
environmental  factors  (e.g.,  noise, 
vibration,  Illumination,  temperature, 
humdity,  toxic  fumes/gases;  as 
appropriate  based  on  findings  of 
previous  testing, 

l.  Prepare  listing  and  descriptions  of 
preliminary  findings  for  scoring 
conference, 

m.  Prepare  outlines  of  OTEA  test 
report  sections  and  ARI  MANPRINT 
report, 

n.  Conduct  preliminary  MANPRINT 
findings  scoring  conference, 

o.  Manage  MANPRINT  specialist 
manpower  staffing.  Prepare 
budget  and  expense  data, 

p.  Video  tape  record  major 
MANPRINT  findings. 


g.  Budget  and  expense  data. 

h.  Video  recording  of  major 
MANPRINT  findings. 
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Table  57 


Major  Operational  last  and  Evaluation  Tasks  for  the  MANPRINT  Assessment  of 
Systems  (Cont'd.) 


RESULTING  DOCUMENTATION 
TASK  LISTING  (Comments) 


TEST  CONDUCT  (cont'd) 


4.  Attend  side  tests  and  4.  a.  Completed  data  collection- 

demonstrations:  forms  and  comment  listings. 

a.  Attend  system  side  test  b.  Videotape  recordings  of 

and  demonstrations,  MANPRINT  experimentation 

(1)  Observe  and  findings. 

(2)  Interview 

b.  Conduct  human  performance  side 
tests, 

(1)  implement  experimental  designs 

(2)  Coordinate  activities  with 
test  directorate 

(3)  Direct  field  experimentation, 

(4)  Collect  data, 

(5)  Submit  data  to  ADP  or 
analyze  data, 

(6)  Videotape  record  side  tests. 


POSTTEST 

1.  Analyze  and  statistically  summarize  !•  The  OTEA  test  report  sections, 

data. 

2.  Prepare  output  formats  for  all  reports.  2.  The  ARI  MANPRINT  test 

assessment  report(s): 


3.  Present  MANPRINT  findings  at  a  formal 
scoring  conference. 


4.  Attend  end  of  test  RAM,  system 
performance,  and  MANPRINT  DAG 
meetings. 

5.  Prepare  data  for  C2E  data  base. 

6.  Write  report  sections  and  full  reports. 


a.  Human  engineering, 

b.  Safety/health  hazards, 

c.  Training, 

d.  Manpower, 

e.  Soldier  characteristics, 

f .  C2E  baseline  prediction 
assessment  and  validation. 

g.  MANPRINT  baseline  pre¬ 
diction  assessment  and 
validation. 


system  acquisition  process  to  insure  that  problems  are  resolved  and  do  not 
continue  into  a  subsequent  phase  (Tragesser,  1985). 

Since  MANPRINT-style  OT&E  is  more  labor  intensive,  and  has  more  pretest 
planning,  there  is  a  need  for  an  earlier  preplanning  start-up  time.  The  very 
need  for  more  pretest  time  runs  counter  to  the  tendency  of  the  weapon  system 
acquisition  process  to  become  accelerated.  The  Impact  of  the  acceleration  can 
be  seen  throughout  OT&E,  and  has  had  an  adverse  effect  on  the  quality  of  the 
resulting  products. 


The  Test  Plan 

The  Sgt  York  test  plan  was  spontaneous.  The  test  plan  was  changed, 
modified,  and  reshaped  on  a  daily  basis  throughout  the  Force-on-Force  phase 
of  FOE  I.  The  inability  to  establish  and  fix  a  test  plan  prior  to  the 
conduct  of  FOE  I  was  a  significant  limitation.  Instrumentation  used  during 
the  test  Introduced  unmeasured  and  uncontrolled  variables  (Seven,  1987). 
Integration  of  the  Combat  Development  Experimentation  Center  (CDEC)  and  1553 
data  bus  was  a  major  problem,  as  well  as  the  problem  to  convert  Real  Time 
Casualty  Assessment  (RTCA),  Multiple  Integrated  Laser  Engagement  System 
(MILES),  and  1553  data  bus  recordings  from  digital  to  analog  form.  Of 
considerable  concern  were  the  Instrumentation  and  data  collection  instruments 
which  were  not  designed  for  human  factors  evaluation,  but  had  to  be  used  for 
the  human  factors  evaluation.  The  instrumentation  constrained  each  trial  to 
approximately  a  30-minute  window  of  data  collection.  The  Instrumentation 
played  a  major  role  in  warping  the  test  plan.  From  this  experience,  it 
becomes  clear  that  test  personnel  must  be  knowledgeable  of  the  Instrumentation 
to  be  used  prior  to  the  development  of  the  test  plan  (note  Table  57),  and  that 
the  test  director  should  not  start  the  test  without  resolution  of  the  types  of 
problems  encountered  during  FOE  I. 

The  instrumentation  testing  constraints  limited  trials  to  approximately 
30-minute  durations  and  signified  a  departure  from  the  originally  planned 
trials  of  longer  duration.  The  resulting  short  trial  periods  created  a 
"snapshot"  view  of  the  operation.  Even  if  the  Instrumentation  would  have 
accommodated  trials  of  24-hour  duration,  the  manning  and  staffing  of  such 
trials  would  not  have  been  feasible.  Trials  of  limited  duration  were  found  to 
distort  measurements,  e.g.,  those  for  Mission  Oriented  Protective  Posture 
(MOPP).  Realistic  trial  times  would  establish  a  more  reliable  data  base  for 
system  evaluation.  Instrumentation  requires  modification  to  accommodate 
longer  trial  times. 

Other  pretest  planning  activities  that  have  proven  valuable  have  been  the 
integration  of  the  human  factors  data  base  into  the  overall  (OTEA)  data  base. 
For  example,  the  critical  task  analysis  data  base  for  timed  measures  should  be 
designed  as  part  of  the  pretest  activities.  Worksheets  should  be  prepared 
that  identify  the  purpose  of  the  measure,  the  desired  data  elements,  the  data 
element  operational  definition,  and  a  description  of  the  planned  analyses. 

The  pretest  activities  should  also  Include  integrating  the  critical  task 
measures  into  the  OTEA  test  data  base  along  with  data  collection  requirements. 
Computer  specification  worksheets  should  include  descriptions  of  data 
elements,  data  display  matrices  for  review,  program  format  and  output 
requirements,  and  examples  of  graphic  displays.  Critical  task  measures  can  be 
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incorporated  in  the  system  performance  and  reliability,  availability,  and 
maintainability  (RAM)  data  collection. 

During  the  Force-on-Force  phase  of  FOE  I,  safety  limitations  impinged  on 
the  day  to  day  conduct  of  the  test.  Units  were  forced  to  modify  safety 
procedures  throughout  data  collection.  Policy  positions  regarding  safety 
procedures  were  frequently  reversed.  An  illustration  of  this  problem  was  the 
issue  of  operating  in  the  heads-out  position.  The  policy  on  heads-out  was 
revised  and  reversed  on  various  occasions.  The  issue  of  laser  safety  was  not 
clearly  evaluated  or  ever  resolved.  Follow-on  evaluation  procedures  which  do 
not  take  into  account  and  do  not  resolve  known  safety  issues  prior  to  test 
conduct  can  easily  confound  the  established  measurement  plan,  the  data 
collection,  and  the  results. 

Many  more  examples  could  be  provided  regarding  contaminated  measurements 
resulting  from  test  plans.  For  example,  the  frequency  of  data  collection  in 
some  test  plans  other  than  the  Sgt  York  test  plan  indicated  that  some 
measurements  were  to  be  taken  so  often  that  the  measurements  would  add  no  new 
knowledge,  and  would  inhibit  participants  from  providing  useful  responses 
throughout  the  data  collection  period.  Other  problems  were  encountered  during 
FOE  I  where  repetitious  scenarios  were  found  to  allow  crews  to  respond  to 
previously  learned  terrain.  Scenarios  that  did  not  truly  differentiate  among 
delay,  attack,  and  road  march  due  to  their  overly  familiar  features  were  not 
of  great  diagnostic  value. 

Some  of  the  potential  problems  encountered  in  the  test  planning  phase  may 
be  induced  by  the  need  for  test  personnel  to  excel.  The  objective  is  to 
design  a  test  plan  that  measures  and  integrates  the  MANPRINT  data,  yet  avoids 
establishing  an  unrealistic  and  overambitious  test  plan.  Accomplishing  both 
objectives  simultaneously  is  not  an  easy  task. 


Data  Analyses  and  Interpretation 


Hard  data  were  accumulated  from  many  sources  during  FOE  I.  The  thru- 
sight  and  crew  compartment  videos  were  of  special  interest  for  data  analysis. 
The  collection  of  data  which  included  video  had  been  anticipated  as  leading  to 
a  richer  data  base  for  further  data  analyses  at  end  of  test.  These  two 
sources  of  video  documentation  proved  to  be  a  disappointment.  Because  of  the 
customary  time  constraints  to  generate  an  OT&E  report  within  a  short  period  of 
time,  it  was  not  feasible  to  review  or  analyze  the  thru-sight  or  crew 
compartment  video  recordings  at  that  time.  It  was  anticipated  that  the  videos 
would  be  useful  during  the  present  study.  Subsequent  review  of  the  video 
recordings  was  accomplished  by  comparing  Time  to  Fire  on  the  1553  data  bus 
along  with  the  video  recordings,  and  the  audio  transcripts.  The  comparison 
revealed  that  the  audio  transcripts  were  deficient  in  reporting  out  complete 
conversations  that  were  observed  on  the  crew  compartment  videos.  However, 
observation  of  the  crew  communication  on  the  videos  did  not  provide  enough 
information  or  insight  into  why  certain  crew  actions  were  or  were  not  taken. 
When  video  recordings  were  compared  to  the  1553  data  base,  it  was  discovered 
that  the  1553  data  base  was  far  superior  and  more  precise  than  the  videos  in 
reporting  crew  activities.  The  videos  did  not  provide  enough  information  for 
analysis.  There  were  also  many  technical  problems  with  the  resulting  videos 
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where  Che  picture  was  so  poor  that  it  could  not  be  deciphered.  In  many 
instances,  the  video's  footage  was  blank. 

Video  techniques  for  measuring  and  analyzing  crew  performance  currently 
have  serious  limitations.  The  analyses  are  expensive  and  lead  to  a  paucity  of 
interpretable  data.  If  video  capability  is  to  be  used,  and  it  is  recommended 
as  a  useful  technique  for  certain  types  of  recording  and  reporting,  it  better 
serves  the  test  community  by  documenting  and  supporting  test  findings.  In  our 
evolving  electronic  age,  individuals  are  being  shaped  by  television  and  video 
to  expect  a  visual  executive  summary  of  current  events.  The  video  OT&E  report 
provides  a  "quick"  overview  of  problems  and  solutions  identified  during  OT&E. 
It  is  a  most  useful  technique  for  disseminating  information  on  design  flaws 
to  design  engineers.  The  test  community  could  benefit  by  producing  OT&E  video 
reports.  Video  reports  should  be  budgeted  and  integrated  into  the  preplanning 
test  activities,  and  into  the  test  budget. 


Realistic  Test  Environment 


Since  test  personnel  try  to  present  a  realistic  picture  of  system 
performance,  limitations  should  not  be  placed  on  crew  members  that  result  in 
crew  inefficiency.  Some  examples  of  the  limitations  placed  on  the  Sgt  York 
crew  members  which  resulted  in  crew  Inefficiency  were  associated  with:  (1) 
Instrumentation  and  data  collection  requirements  which  introduced  uncon¬ 
trollable  and  unmeasurable  variables  into  the  test  design,  (2)  the  continual 
effort  during  FOE  I  to  retrofit  and  upgrade  fire  units  to  full  production 
standards,  and  (3)  limiting  crews  to  operating  in  abnormal  conditions  which  do 
not  reflect  realistic  operation  of  the  weapon  system. 

During  pretest  planning,  problems  with  experimental  and  unproven  in¬ 
strumentation  should  be  resolved  prior  to  conducting  valid  test  trials. 
Instrumentation  which  constrains  the  test  to  the  point  of  impacting  data 
collection  and  subsequent  interpretation  of  the  data  may  generate  ambiguous 
findings.  To  illustrate  this  point,  Sgt  York  crew  members  were  required  to 
continue  expending  rounds  at  dead  targets  because  aircraft  which  the  fire 
units  engaged  displayed  no  realistic  signature.  Measurements  for  crew 
activity  performance  times  would  be  affected  by  this  type  of  constraint.  It 
follows  that  performance  times  for  Fire  Duration  and  Total  Engagement  Duration 
would  be  needlessly  extended.  Trigger  pull  times  associated  with  the  lack  of 
ammunition  depletion  feedback  did  not  represent  the  actual  crew  capability. 

It  is  doubtful  that  the  data  reduction  process  incorporated  methoas  for 
crediting  crews  with  reduced  performance  times  for  Fire  Duration  and  Total 
Engagement  Duration. 

Accelerated  weapon  acquisition  programs  may  tend  to  field  equipment 
before  the  equipment  is  actually  ready  to  go  to  test.  Retrofitting  weapon 
systems  during  training  and  testing  creates  a  situation  where  the  measurement 
and  interpretation  of  the  performance  data  is  questionable.  For  example, 
during  FOE  I,  contractor  retrofit  changes  were  made  to  both  hardware  and 
software.  The  changes  impacted  on  the  Sgt  York  crews,  RAM,  performance 
measures,  and  training.  The  objective  was  to  upgrade  the  equipment  to  full 
production  standards,  but  retrofitting  prior  to  training  and  testing  would 
have  been  preferable.  The  adaptability  of  crew  members  to  adjust  to  on-going 
retrofit  is  an  admirable  trait.  However,  retrofitting  during  training 
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invalidates  training  measures.  Retrofitting  during  testing  Invalidates 
testing  results.  Depending  on  the  nature  of  the  retrofit,  it  could  modify 
crew  action  and  decision  times,  as  well  as  system  processing  times.  There  is 
a  potential  for  the  retrofit  to  modify  the  performance  times  anywhere  along 
the  target  engagement  sequence  continuum.  Performance  times  may  have  been 
distorted  each  time  a  retrofit  occurred.  Retrofitting  contaminated  the  data. 

Artificial  limits  placed  on  the  operation  of  the  equipment  during  testing 
represents  a  serious  threat  to  the  test  design,  measures,  and  interpretation 
of  the  data.  Artificial  limits  run  counter  to  the  design  of  OT&E  plans  which 
strive  to  establish  a  realistic  portrayal  of  system  performance.  Yet 
artificial  limits  have  been  known  to  exist  where  crews  were  requested  to 
perform  under  adverse  conditions.  To  illustrate  an  artificial  limitation 
during  the  FOE  I,  Sgt  York  crews  were  forced  to  operate  the  fire  unit  under 
constraints  that  would  run  counter  to  crew  training,  and  counter  to  the  design 
and  function  of  the  fire  unit.  The  fire  unit  was  designed  to  have  multiple 
system  mode  options  during  the  target  engagement  sequence  for  acquisition  of 
the  target.  However,  during  a  FOE  I  trial,  Sgt  York  crews  were  required  to 
obscure  their  gunslghts  with  covers  in  order  to  simulate  a  night  environment. 
Subsequently,  lasers  were  declared  non-operational  so  that  only  radar 
acquisition  modes  could  be  used  in  parts  of  the  test. 

OT&E  is  supposed  to  measure  performance  under  conditions  of  maximum 
fidelity  to  the  operational  environment  as  much  as  possible.  According  to 
Meister  (1987),  the  goal  of  testing  is  to  simulate  the  operational  system  and 
environment,  not  to  distort  the  operational  environment.  Continual  efforts 
should  be  made  to  strive  for  fidelity  in  the  test  environment. 


Personnel  Characteristics/Training/Performance  Measures 

Requirements  for  military  personnel  serving  as  subjects  during  OT&E  most 
frequently  involve  the  need  for  subjects  to  have  both  training  and  experience. 
During  FOE  1,  not  all  subjects  received  individual  training.  Those  subjects 
who  did  not  receive  Sgt  York  individual  training  were  considered  experienced 
Sgt  York  crew  members,  in  that  they  had  previous  experience  with  fire  units. 
The  decision  to  equate  the  individuals  with  previous  Sgt  York  experience  with 
those  who  completed  individual  training  was  perhaps  driven  by  the  need  to 
increase  the  pool  of  candidate  subjects.  Those  individuals  who  did  receive 
individual  training  found  that  the  training  schedule  was  abbreviated.  When 
the  two  groups  of  individuals  were  compared,  it  was  not  possible  to  dif¬ 
ferentiate  the  effect  of  training.  For  that  matter,  it  was  not  possible  to 
relate  performance  measures  to  subject  characteristics,  and  selection  scores 
such  as  height,  weight,  age,  level  of  education,  ASVAB  scores,  etc. 

Although  the  subjects  were  representative  of  the  individuals  who  were 
supposed  to  operate  the  fire  units  eventually,  the  number  of  subjects  was 
small.  Working  under  the  constraint  of  a  small  sample  size  precludes 
statistical  analyses  to  correlate  Individual  characteristics  and  selection 
scores  to  performance  measures. 
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Dissemination  of  Information  from  OT&E 


In  a  list  of  behavioral  questions  associated  with  system  development 
suggested  by  Meister  (1987),  the  last  item  was  "Evaluate  and  recommend  system 
improvement  modifications."  Once  improvements  have  been  identified  and 
specified  in  report  form,  other  methods  for  dissemination  of  the  identified 
Improvements  should  be  sought.  For  example,  how  can  OT&E  personnel  dissemi¬ 
nate  testing  results  in  an  effective  way  so  that  the  data  can  be  applied 
effectively  to  modify  a  current  system  or  improve  future  systems?  How  can  the 
data  be  made  accessible  and  useful  for  design  engineers?  To  illustrate  this 
problem,  the  Sgt  York  fire  unit  incorporated  the  M48  chassis  as  part  of  the 
weapon  system.  The  M48  chassis  is  a  Korean  vintage  chassis  with  previously 
documented  deficiencies.  There  are  many  examples  in  OT&E  where  equipment 
deficiencies  have  been  identified,  and  where  design  engineers  continue  to 
replicate  their  mistakes  in  new  designs.  There  does  not  seem  to  be  a 
corporate  history  or  military  record  or  a  data  repository  where  individuals 
may  seek  information  and  learn  from  previous  mistakes. 

Innovative  attempts  are  being  made  to  assist  in  alleviating  the  problem 
of  "repeated  design  failures."  MANPRINT  is  one  such  attempt  where  the 
acquisition  process  is  to  be  halted  at  each  review  stage  until  identifiable 
problems  are  resolved.  Another  attempt  to  influence  the  design  stage  is 
being  undertaken  by  the  U.S.  Air  Force,  as  the  lead  service  for  the  Depart¬ 
ment  of  Defense.  The  proposed  project  is  commonly  known  as  CSERIAC  or  Crew 
Station  Ergonomic  Information  Analysis  Center.  At  the  present  time,  this  is  a 
concept,  but  not  an  implemented  program.  CSERIAC  has  the  potential  to 
establish  a  center  that  will  allow  designers  the  opportunity  to  access  data 
which  is  technically  sound  and  will  improve  future  systems  prior  to  the 
production  of  prototypes.  The  OT&F  video  reports  previously  discussed  would 
also  be  quite  useful  to  design  engineers  to  assist  in  alleviating  the  problem 
of  repeating  mistakes.  Questions  of  dissemination  concern  who  should  receive 
the  OT&E  information,  in  what  format,  and  usin~  what  administrative  vehicle. 
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Appendix  A 

Description  of  Data  Files 


The  purpose  of  this  appendix  is  to  describe  the  data  files  and  data 
processing  for  the  analyses  in  this  study.  The  data  files  and  their  rela¬ 
tionships  are  shown  in  Figure  A-l.  Each  of  these  data  files  are  available  on 
computer  disk  suitable  for  MSDOS  and  IBM-compatible  personal  computers.* 

The  primary  files  include  an  engagement  data  base  (ENG2),  a  crew  composi¬ 
tion  data  base  (CREWCOMP),  a  personnel  description  data  base  (PERSDESC),  and  a 
scenario  description  data  base  (SCENDESC).  These  were  created  using  the 
Ashton-Tate  dBASE  data  base  management  software  (Ashton-Tate,  1985).  The 
distribution  disk  includes  files  for  both  dBASE  II  and  dBASE  III  versions  of 
this  software  (with  file  extensions  .db2  and  .db3  respectively;  the  ap¬ 
propriate  version  must  be  renamed  to  .dbf  before  use  with  dBASE).  The 
structure  and  content  of  these  files  are  defined  in  Tables  A-l  through  A-4. 

The  tables  show  the  field  name,  type  of  data  (Ocharacter ,  N»numeric),  the 
number  of  decimal  places  if  numeric  data,  and  a  full-text  version  of  the 
field  name. 

The  ENG2  data  base  is  a  corrected  version  of  the  initial  data  entry  file 
(not  included  on  the  distribution  disk).  The  dBASE  software  can  operate  under 
program  control,  as  well  as  with  on-line  interactive  operator  inputs.  For 
example,  COMPUTE. PRG  is  a  program  which  computes  the  quantities  shown  in  Table 
3  of  the  main  report.  These  results  were  later  stored  as  TOTAL . DAT  for 
subsequent  statistical  analyses. 

The  dBASE  software  permits  using  these  data  bases  in  combination,  relating 
the  appropriate  data  in  one  file  to  that  in  another.  For  example,  the  data  in 
the  CREWCOMP  file  can  be  used  to  identify  the  specific  crew  member  who  took 
action  in  the  ENG2  file,  compute  performance  measures,  and  create  a  file  of 
individual  performance  (INDPERF,  structure  shown  in  Table  A-5).  This  was 
accomplished  using  the  COMBINE. PRG  and  C0MPUTE2.PRG  programs.  Although  the 
information  can  be  derived  whenever  desired  from  the  basic  data  base  files,  it 
was  more  efficient  to  create  a  permanent  INDPERF  file  for  subsequent  repeated 
use.  A  file  named  INDPERF.DAT  is  an  ASCII  format  file  used  for  input  to 
statistical  analysis  software. 

The  TOTAL  and  INDPERF  files  were  converted  to  a  form  required  for 
statistical  analysis.  The  statistical  package  used  is  SYSTAT  (Wilkinson, 
1987).  The  files  needed  are  simply  ASCII  files  which  are  transformed  by  the 
DATA  module  to  files  which  are  used  by  the  remaining  SYSTAT  modules.  These 
files  are  included  in  the  distribution  disk  with  the  extension  .SYS.  If  the 
ASCII  files  are  to  be  used  for  a  statistical  package  other  than  SYSTAT,  it 
should  be  noted  that  missing  data  are  coded  in  the  dBASE  files  with  the  value 
999.  No  other  data  have  values  this  large.  Under  the  SYSTAT  system,  missing 
data  are  recoded  as 


*This  appendix  was  prepared  by  Richard  W.  Obermayer,  Vreuls  Research 
Corporation. 
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!  ENG2.DB2  ! 
!  ENG2.DB3  ! 


!  CREW COMP. DB2 ! 
JCREWCOMP.DB3! 


! PERSDESC. DB2 ! 
JPERSDESC.DB3! 


!  SCENDESC. DB2 ! 
!  SCENDESC.DB3 ! 


H - h  H - 1- 

i  i  ( 

!  4 - (. 

!  ! 

V  V 

+———— - (•  4-— ————4 

!  COMPUTE . DAT  !  !  COMBINE. PRG  ! 

H - 1-  !  COMPUTE2 .PRG  ! 

i  -  ■  -+ 

!  ! 

V  V 

H - h  H - h 

!  TOTAL.DAT  !  !  INDPERF . DBF  ! 

4 - H  !  INAT  ! 

I  4 — - t- 

!  i 

V  V 


4 


r 

! 

V 


DBASE  II  OR  DBASE  III 
DATA  BASE  MANAGEMENT 
SYSTEM  SOFTWARE 


i 

V 


j 

V 


4 - f 

!  MISDAT.CMD  ! 

4 - h- 

I 

j 

V 

4— — - — t- 

!  TOTALS. SYS  ! 

4. - (- 

J 

V 


!  MISDAT2.CMD  ! 

4 - 

! 

j 

V 

4 - 1- 

!  INDPERF. SYS  ! 

4 - 

J 

V 


J  STATISTICAL  ANALYSIS  ! 
!  (DESCRIPTIVE  STATISTICS,  ! 
!  BOX-AND-WHISKER  DIAGRAMS,  ! 
!  CORRELATION,  ANOVA,  ! 
!  MULTIPLE  REGRESSION)  ! 
4 - H 


SYSTAT 

STATISTICAL 

ANALYSIS 

SOFTWARE 


Figure  A-l.  Data  processing  flow  diagram. 
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Table  A-l 


STRUCTURE  FOR  FILE:  A:ENG2.DBF 
NUMBER  OF  RECORDS:  00271 


FLD 

NAME 

TYPE 

WIDTH 

DEC 

REMARKS 

001 

TRIAn 

C 

004 

TRIAL  NUMBER 

002 

FU 

C 

001 

FIRE  UNIT 

003 

ENGAGE 

C 

002 

ENGAGEMENT 

004 

HR 

N 

002 

HOUR 

005 

MIN 

N 

002 

MINUTE 

006 

TPRES 

N 

005 

001 

PRESENTATION  TIME 

007 

TGTID 

C 

012 

TARGET  IDENTIFICATION 

008 

RANGE 

N 

004 

001 

TGT  RANGE  (KILOMETERS) 

009 

NUMTGT 

N 

002 

NUMBER  OF  TGTS  PRESENTED 

010 

TGTTYPE 

C 

002 

TARGET  TYPE  (FIXED/ROTARY) 

011 

SYSMODE 

C 

036 

SEQUENCE  OF  SYSTEM  MODES 

012 

USEMODE 

c 

001 

MODE  USED 

013 

PCREW 

c 

002 

CREWMEMBER  POINTING 

014 

PDEP 

N 

005 

001 

POINTER  DEPRESS  TIME 

015 

PREL 

N 

005 

001 

POINTER  RELEASE  TIME 

016 

LDEP 

N 

005 

001 

LAZER  DEPRESS  TIME 

017 

LREL 

N 

005 

001 

LAZER  RELEASE  TIME 

018 

RCON 

N 

005 

001 

RADAR  CUE  ON  TIME 

019 

LCON 

N 

005 

001 

LAZER  CUE  ON  TIME 

020 

FCBLINK 

N 

005 

001 

FIRE  CUE  BLINKING 

021 

FCSOLID 

N 

005 

001 

FIRE  CUE  SOLID 

022 

TRIGON 

N 

005 

001 

TRIGGER  ON  TIME 

023 

TCREW 

C 

002 

CREWMEMBER  FIRING 

024 

TSWITCH 

C 

001 

TRIGGER  SWITCH  (L/R) 

025 

TRIGOFF 

N 

005 

001 

TRIGGER  OFF  TIME 

026 

TERM 

N 

005 

001 

TERMINATE  TIME 

Note. 


HR  and  MIN  are 
entered  as  the 


starting  time  hour  and  minute;  other  times  are 
number  of  seconds  after  HR  and  MIN. 
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Table  A-2 


STRUCTURE  FOR  FILE:  A.'CREWCOMP.DBF 
NUMBER  OF  RECORDS:  00113 


FLD 

NAME 

TYPE 

WIDTH 

DEC  REMARKS 

001 

TRIAL 

C 

TRIAL  NUMBER 

002 

DATE 

C 

008 

DATE 

003 

SCEN 

C 

SCENARIO 

004 

FU 

C 

FIRE  UNIT 

005 

GNO 

C 

GUNNER  ID 

006 

SNO 

C 

SQUAD  LEADER  ID 

007 

DNO 

C 

DRIVER  ID 

Table  A-3 


STRUCTURE  FOR  FILE:  A:PERSDESC.DBF 
NUMBER  OF  RECORDS:  00015 


FLD 

NAME 

TYPE 

WIDTH 

DEC 

REMARKS 

001 

IDNO 

C 

004 

IDENTIFICATION  NR. 

002 

RANK 

C 

002 

RANK 

003 

POS 

C 

001 

POSITION  (S,G,D) 

004 

HT 

N 

002 

HEIGHT 

005 

WT 

N 

003 

WEIGHT 

006 

OF 

N 

003 

OF  SCORE 

007 

EL 

N 

003 

EL  SCORE 

008 

AFQT 

N 

002 

AFQT  SCORE 

009 

CAT 

C 

004 

AFQT  CATEGORY 

010 

GT 

N 

003 

GT  SCORE 

Oil 

SYEXP 

C 

003 

SGT  YORK  EXPERIENCE 

012 

INDTRNG 

N 

004 

001 

INDIVID.  TRAINING 

013 

COLTRNG 

C 

003 

COLLECTIVE  TRAINING 

Table  A-4 


STRUCTURE  FOR  FILE:  A: SCENDESC.DBF 
NUMBER  OF  RECORDS:  00029 


FLD 

NAME 

TYPE 

WIDTH 

DEC  REMARKS 

001 

TRIAL 

C 

004 

TRIAL  NUMBER 

002 

DATE 

C 

008 

DATE 

003 

TYPE 

C 

013 

SCENARIO  TYPE 

004 

NO 

C 

002 

SCENARIO  NUMBER 

005 

DorN 

C 

001 

DAY  OR  NIGHT 

006 

ENDTIME 

N 

004 

ENDING  TIME 

007 

TACTICS 

C 

012 

TACTICS 

008 

MOPP 

C 

003 

MOPP  (YES/NO) 

009 

ECM 

c 

006 

ECM  CONDITION 

Table  A-5 


STRUCTURE  FOR  FILE:  A:INDPERF.DBF 
NUMBER  OF  RECORDS:  00271 


FLD 

NAME 

TYPE 

WIDTH 

DEC 

REMARKS 

001 

TRIAL 

C 

004 

TRIAL  NUMBER 

002 

FU 

C 

001 

FIRE  UNIT 

003 

ENGAGE 

C 

002 

ENGAGEMENT 

004 

HR 

N 

002 

HOUR  (START) 

005 

MTN 

N 

002 

MINUTE  (START) 

006 

TPRES 

N 

005 

001 

PRESENTATION  TIME 

007 

PCREW 

C 

002 

CREWMEMBER  POINTING 

008 

PDEP 

N 

005 

001 

POINTER  DEPRESS  TIME 

009 

PREL 

N 

005 

001 

POINTER  RELEASE  TIME 

010 

LDEP 

N 

005 

001 

LAZER  DEPRESS  TIME 

Oil 

LREL 

N 

005 

001 

LAZER  RELEASE  TIME 

012 

TCREW 

C 

002 

CREWMEMBER  FIRING 

013 

IDNO 

r 

004 

CREWMEMBER  TAKING  FIRST  ACTION 

014 

SELCT 

N 

005 

001 

SELECT  TIME  MEASURE 

015 

PTRON 

N 

005 

001 

POINTER  TIME  MEASURE 

016 

LZRON 

N 

005 

001 

LAZER  TIME  MEASURE 

-5 
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Appendix  B 


Correlations  Among  Parameter  Measures 


As  may  be  seen  throughout  the  discussions  on  research  findings,  five 
basic  parameters  form  the  essence  of  the  measurement  system.  These  parame¬ 
ters  are: 

o  Target  Selection 

o  Target  Acquisition 

o  Time  to  Fire 

o  Fire  Duration 

o  Engagement  Time 

These  parameters  have  been  discussed  and  defined  in  the  chapter  on  human 
performance  data.  Further,  some  of  the  parameters  have  been  decomposed  to 
lower  levels  of  the  process  as  shown  in  Table  3. 

It  is  of  some  interest  to  look  at  the  correlations  among  the  basic  five 
measures.  These  correlations  are  shown  in  Table  63  as  Pearson  product  moment 
correlations. 


Table  B-l 

Pearson  Correlation  Among  Parameters 


Parameters 

|  Select 

I  Acquire 

I  Time  to  | 
Fire 

Fire  ! 

Duration 

I  Engagement 
Time 

Select 

1.000 

Acquire 

-0.079 

1.000 

Time  to  Fire 

0.843 

0.276 

Fire  Duration 

0.000 

-0.071 

1.000 

Engagement  Time 

0.707 

0.189 

0.832 

0.560 

1.000 

The  patterns  seen  in  Table  B-l  are  exactly  as  might  be  expected  from  a 
close  examination  of  the  actual  processes  as  they  were  performed.  At  first 
look,  one  might  predict  a  high  and  growing  correlation  among  all  these 
measures  since  they  are  sequentially  dependent  combining  to  form  a  target 
engagement  sequence.  But,  as  Table  63  shows,  this  is  not  necessarily  the 
case.  So,  for  example,  acquisition  times,  based  so  strongly  on  automatic 
data  processing,  show  little  variation  with  respect  to  Fire  Duration  and 
Engagement  Time.  This  subfunction  is  relatively  fixed  with  regard  to  the 
process. 
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Target  Selection  Times,  on  the  other  hand,  vary  considerably  and  have  a 
very  strong  effect  on  subsequent  Time  to  Fire  (r  ■  .843)  and  Total  Engagement 
Time  (r  -  .707).  Fire  Duration  presents  a  different  kind  of  pattern  - 
strongly  influencing  engagement  time  (r  -  .560)  but  not  related  to  Selection, 
Acquisition,  or  Time  to  Fire. 

A  technical  problem  in  most  models  is  the  degree  of  orthogonality  among 
various  system  parameters.  With  complex  multi-level  systems  involving  the 
human  operator  in  a  variety  of  ways  from  total  immersion  to  exclusion,  the 
correlations  among  parameters  and  their  measures  will  vary  as  do  those  in 
Table  63.  In  short,  some  parameters  are  independent  of  the  other  process 
parameters  and  many  others  are  not.  This  fact,  of  course,  will  complicate 
model  development  as  well  as  data  interpretation. 
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